2023-11-28 03:53:14,325 INFO [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.mob.compactions.TestMobCompactor timeout: 13 mins 2023-11-28 03:53:14,578 DEBUG [main] hbase.HBaseTestingUtility(342): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/2f2fe2ad-19b7-95b4-b267-35017f394f29 2023-11-28 03:53:14,594 INFO [Time-limited test] hbase.HBaseTestingUtility(1068): Starting up minicluster with option: StartMiniClusterOption{numMasters=1, masterClass=null, numRegionServers=1, rsPorts=, rsClass=null, numDataNodes=1, dataNodeHosts=null, numZkServers=1, createRootDir=false, createWALDir=false} 2023-11-28 03:53:14,594 INFO [Time-limited test] hbase.HBaseZKTestingUtility(82): Created new mini-cluster data directory: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/2f2fe2ad-19b7-95b4-b267-35017f394f29/cluster_c657509c-448c-2cac-5018-4e5cd83a6dc3, deleteOnExit=true 2023-11-28 03:53:14,595 INFO [Time-limited test] hbase.HBaseTestingUtility(1082): STARTING DFS 2023-11-28 03:53:14,596 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting test.cache.data to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/2f2fe2ad-19b7-95b4-b267-35017f394f29/test.cache.data in system properties and HBase conf 2023-11-28 03:53:14,596 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.tmp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/2f2fe2ad-19b7-95b4-b267-35017f394f29/hadoop.tmp.dir in system properties and HBase conf 2023-11-28 03:53:14,597 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.log.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/2f2fe2ad-19b7-95b4-b267-35017f394f29/hadoop.log.dir in system properties and HBase conf 2023-11-28 03:53:14,597 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.local.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/2f2fe2ad-19b7-95b4-b267-35017f394f29/mapreduce.cluster.local.dir in system properties and HBase conf 2023-11-28 03:53:14,598 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.temp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/2f2fe2ad-19b7-95b4-b267-35017f394f29/mapreduce.cluster.temp.dir in system properties and HBase conf 2023-11-28 03:53:14,598 INFO [Time-limited test] hbase.HBaseTestingUtility(759): read short circuit is OFF 2023-11-28 03:53:14,724 WARN [Time-limited test] util.NativeCodeLoader(62): Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2023-11-28 03:53:15,153 DEBUG [Time-limited test] fs.HFileSystem(308): The file system is not a DistributedFileSystem. Skipping on block location reordering 2023-11-28 03:53:15,160 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-labels.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/2f2fe2ad-19b7-95b4-b267-35017f394f29/yarn.node-labels.fs-store.root-dir in system properties and HBase conf 2023-11-28 03:53:15,161 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-attribute.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/2f2fe2ad-19b7-95b4-b267-35017f394f29/yarn.node-attribute.fs-store.root-dir in system properties and HBase conf 2023-11-28 03:53:15,161 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.log-dirs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/2f2fe2ad-19b7-95b4-b267-35017f394f29/yarn.nodemanager.log-dirs in system properties and HBase conf 2023-11-28 03:53:15,162 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/2f2fe2ad-19b7-95b4-b267-35017f394f29/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-11-28 03:53:15,162 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.active-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/2f2fe2ad-19b7-95b4-b267-35017f394f29/yarn.timeline-service.entity-group-fs-store.active-dir in system properties and HBase conf 2023-11-28 03:53:15,163 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.done-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/2f2fe2ad-19b7-95b4-b267-35017f394f29/yarn.timeline-service.entity-group-fs-store.done-dir in system properties and HBase conf 2023-11-28 03:53:15,163 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/2f2fe2ad-19b7-95b4-b267-35017f394f29/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-11-28 03:53:15,166 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/2f2fe2ad-19b7-95b4-b267-35017f394f29/dfs.journalnode.edits.dir in system properties and HBase conf 2023-11-28 03:53:15,168 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.datanode.shared.file.descriptor.paths to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/2f2fe2ad-19b7-95b4-b267-35017f394f29/dfs.datanode.shared.file.descriptor.paths in system properties and HBase conf 2023-11-28 03:53:15,168 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting nfs.dump.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/2f2fe2ad-19b7-95b4-b267-35017f394f29/nfs.dump.dir in system properties and HBase conf 2023-11-28 03:53:15,169 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting java.io.tmpdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/2f2fe2ad-19b7-95b4-b267-35017f394f29/java.io.tmpdir in system properties and HBase conf 2023-11-28 03:53:15,169 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/2f2fe2ad-19b7-95b4-b267-35017f394f29/dfs.journalnode.edits.dir in system properties and HBase conf 2023-11-28 03:53:15,169 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.provided.aliasmap.inmemory.leveldb.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/2f2fe2ad-19b7-95b4-b267-35017f394f29/dfs.provided.aliasmap.inmemory.leveldb.dir in system properties and HBase conf 2023-11-28 03:53:15,170 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting fs.s3a.committer.staging.tmp.path to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/2f2fe2ad-19b7-95b4-b267-35017f394f29/fs.s3a.committer.staging.tmp.path in system properties and HBase conf Formatting using clusterid: testClusterID 2023-11-28 03:53:15,860 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-11-28 03:53:15,865 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-11-28 03:53:16,231 WARN [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties 2023-11-28 03:53:16,432 INFO [Time-limited test] log.Slf4jLog(67): Logging to org.slf4j.impl.Reload4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog 2023-11-28 03:53:16,453 WARN [Time-limited test] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-11-28 03:53:16,498 INFO [Time-limited test] log.Slf4jLog(67): jetty-6.1.26 2023-11-28 03:53:16,541 INFO [Time-limited test] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/hdfs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/2f2fe2ad-19b7-95b4-b267-35017f394f29/java.io.tmpdir/Jetty_localhost_44519_hdfs____.cmd7eh/webapp 2023-11-28 03:53:16,695 INFO [Time-limited test] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:44519 2023-11-28 03:53:16,706 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-11-28 03:53:16,706 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-11-28 03:53:17,274 WARN [Listener at localhost/34103] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-11-28 03:53:17,364 WARN [Listener at localhost/34103] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-11-28 03:53:17,387 WARN [Listener at localhost/34103] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-11-28 03:53:17,394 INFO [Listener at localhost/34103] log.Slf4jLog(67): jetty-6.1.26 2023-11-28 03:53:17,399 INFO [Listener at localhost/34103] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/2f2fe2ad-19b7-95b4-b267-35017f394f29/java.io.tmpdir/Jetty_localhost_37889_datanode____kjwxbw/webapp 2023-11-28 03:53:17,505 INFO [Listener at localhost/34103] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:37889 2023-11-28 03:53:17,913 WARN [Listener at localhost/42853] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-11-28 03:53:18,445 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xcee27f5d5f9dadc2: Processing first storage report for DS-3e272f40-ba73-4e34-81ee-8c3da9d3c697 from datanode 341a4277-f306-4f80-b29e-d311107d0264 2023-11-28 03:53:18,446 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xcee27f5d5f9dadc2: from storage DS-3e272f40-ba73-4e34-81ee-8c3da9d3c697 node DatanodeRegistration(127.0.0.1:40385, datanodeUuid=341a4277-f306-4f80-b29e-d311107d0264, infoPort=46661, infoSecurePort=0, ipcPort=42853, storageInfo=lv=-57;cid=testClusterID;nsid=1987968935;c=1701143595945), blocks: 0, hasStaleStorage: true, processing time: 1 msecs, invalidatedBlocks: 0 2023-11-28 03:53:18,447 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xcee27f5d5f9dadc2: Processing first storage report for DS-3751b27c-7cb6-435b-9ab0-1a5d92cebad6 from datanode 341a4277-f306-4f80-b29e-d311107d0264 2023-11-28 03:53:18,447 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xcee27f5d5f9dadc2: from storage DS-3751b27c-7cb6-435b-9ab0-1a5d92cebad6 node DatanodeRegistration(127.0.0.1:40385, datanodeUuid=341a4277-f306-4f80-b29e-d311107d0264, infoPort=46661, infoSecurePort=0, ipcPort=42853, storageInfo=lv=-57;cid=testClusterID;nsid=1987968935;c=1701143595945), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-11-28 03:53:18,496 DEBUG [Listener at localhost/42853] hbase.HBaseTestingUtility(649): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/2f2fe2ad-19b7-95b4-b267-35017f394f29 2023-11-28 03:53:18,604 INFO [Listener at localhost/42853] zookeeper.MiniZooKeeperCluster(258): Started connectionTimeout=30000, dir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/2f2fe2ad-19b7-95b4-b267-35017f394f29/cluster_c657509c-448c-2cac-5018-4e5cd83a6dc3/zookeeper_0, clientPort=55641, secureClientPort=-1, dataDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/2f2fe2ad-19b7-95b4-b267-35017f394f29/cluster_c657509c-448c-2cac-5018-4e5cd83a6dc3/zookeeper_0/version-2, dataDirSize=457 dataLogDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/2f2fe2ad-19b7-95b4-b267-35017f394f29/cluster_c657509c-448c-2cac-5018-4e5cd83a6dc3/zookeeper_0/version-2, dataLogSize=457 tickTime=2000, maxClientCnxns=300, minSessionTimeout=4000, maxSessionTimeout=40000, clientPortListenBacklog=-1, serverId=0 2023-11-28 03:53:18,623 INFO [Listener at localhost/42853] zookeeper.MiniZooKeeperCluster(283): Started MiniZooKeeperCluster and ran 'stat' on client port=55641 2023-11-28 03:53:18,635 INFO [Listener at localhost/42853] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-11-28 03:53:18,638 INFO [Listener at localhost/42853] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-11-28 03:53:19,334 INFO [Listener at localhost/42853] util.FSUtils(471): Created version file at hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb with version=8 2023-11-28 03:53:19,334 INFO [Listener at localhost/42853] hbase.HBaseTestingUtility(1406): Setting hbase.fs.tmp.dir to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/hbase-staging 2023-11-28 03:53:19,638 INFO [Listener at localhost/42853] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl 2023-11-28 03:53:20,081 INFO [Listener at localhost/42853] client.ConnectionUtils(126): master/jenkins-hbase4:0 server-side Connection retries=3 2023-11-28 03:53:20,113 INFO [Listener at localhost/42853] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-11-28 03:53:20,114 INFO [Listener at localhost/42853] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-11-28 03:53:20,114 INFO [Listener at localhost/42853] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-11-28 03:53:20,114 INFO [Listener at localhost/42853] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-11-28 03:53:20,114 INFO [Listener at localhost/42853] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-11-28 03:53:20,259 INFO [Listener at localhost/42853] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-11-28 03:53:20,332 DEBUG [Listener at localhost/42853] util.ClassSize(228): Using Unsafe to estimate memory layout 2023-11-28 03:53:20,438 INFO [Listener at localhost/42853] ipc.NettyRpcServer(120): Bind to /172.31.14.131:38593 2023-11-28 03:53:20,448 INFO [Listener at localhost/42853] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-11-28 03:53:20,450 INFO [Listener at localhost/42853] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-11-28 03:53:20,473 INFO [Listener at localhost/42853] zookeeper.RecoverableZooKeeper(93): Process identifier=master:38593 connecting to ZooKeeper ensemble=127.0.0.1:55641 2023-11-28 03:53:20,518 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:385930x0, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-11-28 03:53:20,520 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): master:38593-0x100321f3a150000 connected 2023-11-28 03:53:20,559 DEBUG [Listener at localhost/42853] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-11-28 03:53:20,560 DEBUG [Listener at localhost/42853] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-11-28 03:53:20,563 DEBUG [Listener at localhost/42853] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-11-28 03:53:20,571 DEBUG [Listener at localhost/42853] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=38593 2023-11-28 03:53:20,572 DEBUG [Listener at localhost/42853] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=38593 2023-11-28 03:53:20,572 DEBUG [Listener at localhost/42853] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=38593 2023-11-28 03:53:20,572 DEBUG [Listener at localhost/42853] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=38593 2023-11-28 03:53:20,576 DEBUG [Listener at localhost/42853] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=38593 2023-11-28 03:53:20,582 INFO [Listener at localhost/42853] master.HMaster(444): hbase.rootdir=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb, hbase.cluster.distributed=false 2023-11-28 03:53:20,654 INFO [Listener at localhost/42853] client.ConnectionUtils(126): regionserver/jenkins-hbase4:0 server-side Connection retries=3 2023-11-28 03:53:20,654 INFO [Listener at localhost/42853] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-11-28 03:53:20,655 INFO [Listener at localhost/42853] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-11-28 03:53:20,655 INFO [Listener at localhost/42853] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-11-28 03:53:20,655 INFO [Listener at localhost/42853] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-11-28 03:53:20,655 INFO [Listener at localhost/42853] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-11-28 03:53:20,660 INFO [Listener at localhost/42853] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-11-28 03:53:20,663 INFO [Listener at localhost/42853] ipc.NettyRpcServer(120): Bind to /172.31.14.131:41203 2023-11-28 03:53:20,665 INFO [Listener at localhost/42853] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-11-28 03:53:20,671 DEBUG [Listener at localhost/42853] mob.MobFileCache(121): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-11-28 03:53:20,672 INFO [Listener at localhost/42853] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-11-28 03:53:20,675 INFO [Listener at localhost/42853] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-11-28 03:53:20,677 INFO [Listener at localhost/42853] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:41203 connecting to ZooKeeper ensemble=127.0.0.1:55641 2023-11-28 03:53:20,681 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:412030x0, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-11-28 03:53:20,682 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:41203-0x100321f3a150001 connected 2023-11-28 03:53:20,682 DEBUG [Listener at localhost/42853] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-11-28 03:53:20,683 DEBUG [Listener at localhost/42853] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-11-28 03:53:20,684 DEBUG [Listener at localhost/42853] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-11-28 03:53:20,685 DEBUG [Listener at localhost/42853] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=41203 2023-11-28 03:53:20,686 DEBUG [Listener at localhost/42853] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=41203 2023-11-28 03:53:20,686 DEBUG [Listener at localhost/42853] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=41203 2023-11-28 03:53:20,688 DEBUG [Listener at localhost/42853] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=41203 2023-11-28 03:53:20,689 DEBUG [Listener at localhost/42853] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=41203 2023-11-28 03:53:20,691 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2168): Adding backup master ZNode /hbase/backup-masters/jenkins-hbase4.apache.org,38593,1701143599478 2023-11-28 03:53:20,700 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-11-28 03:53:20,703 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/backup-masters/jenkins-hbase4.apache.org,38593,1701143599478 2023-11-28 03:53:20,724 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-11-28 03:53:20,724 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-11-28 03:53:20,724 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-11-28 03:53:20,726 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-11-28 03:53:20,729 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(227): Deleting ZNode for /hbase/backup-masters/jenkins-hbase4.apache.org,38593,1701143599478 from backup master directory 2023-11-28 03:53:20,729 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-11-28 03:53:20,734 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/backup-masters/jenkins-hbase4.apache.org,38593,1701143599478 2023-11-28 03:53:20,734 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-11-28 03:53:20,734 WARN [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-11-28 03:53:20,735 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(237): Registered as active master=jenkins-hbase4.apache.org,38593,1701143599478 2023-11-28 03:53:20,737 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating data MemStoreChunkPool with chunk size 2 MB, max count 352, initial count 0 2023-11-28 03:53:20,738 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating index MemStoreChunkPool with chunk size 204.80 KB, max count 391, initial count 0 2023-11-28 03:53:20,822 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] util.FSUtils(620): Created cluster ID file at hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/hbase.id with ID: ce5c0691-8473-42df-8a7d-422da22a0638 2023-11-28 03:53:20,866 INFO [master/jenkins-hbase4:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-11-28 03:53:20,881 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-11-28 03:53:20,927 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x3d86cced to 127.0.0.1:55641 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-11-28 03:53:20,959 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.AbstractRpcClient(189): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@5723b48e, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-11-28 03:53:20,982 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-11-28 03:53:20,984 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-11-28 03:53:21,000 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(264): ClientProtocol::create wrong number of arguments, should be hadoop 3.2 or below 2023-11-28 03:53:21,000 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(270): ClientProtocol::create wrong number of arguments, should be hadoop 2.x 2023-11-28 03:53:21,002 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(279): can not find SHOULD_REPLICATE flag, should be hadoop 2.x java.lang.IllegalArgumentException: No enum constant org.apache.hadoop.fs.CreateFlag.SHOULD_REPLICATE at java.lang.Enum.valueOf(Enum.java:238) at org.apache.hadoop.fs.CreateFlag.valueOf(CreateFlag.java:63) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.loadShouldReplicateFlag(FanOutOneBlockAsyncDFSOutputHelper.java:277) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.(FanOutOneBlockAsyncDFSOutputHelper.java:304) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:139) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:855) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2193) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:528) at java.lang.Thread.run(Thread.java:750) 2023-11-28 03:53:21,006 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(243): No decryptEncryptedDataEncryptionKey method in DFSClient, should be hadoop version with HDFS-12396 java.lang.NoSuchMethodException: org.apache.hadoop.hdfs.DFSClient.decryptEncryptedDataEncryptionKey(org.apache.hadoop.fs.FileEncryptionInfo) at java.lang.Class.getDeclaredMethod(Class.java:2130) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelperWithoutHDFS12396(FanOutOneBlockAsyncDFSOutputSaslHelper.java:182) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelper(FanOutOneBlockAsyncDFSOutputSaslHelper.java:241) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.(FanOutOneBlockAsyncDFSOutputSaslHelper.java:252) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:140) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:855) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2193) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:528) at java.lang.Thread.run(Thread.java:750) 2023-11-28 03:53:21,007 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-11-28 03:53:21,041 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7693): Creating {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, under table dir hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/MasterData/data/master/store-tmp 2023-11-28 03:53:21,070 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-28 03:53:21,071 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-11-28 03:53:21,071 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-11-28 03:53:21,071 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-11-28 03:53:21,071 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-11-28 03:53:21,071 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-11-28 03:53:21,071 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-11-28 03:53:21,071 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-11-28 03:53:21,073 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/MasterData/WALs/jenkins-hbase4.apache.org,38593,1701143599478 2023-11-28 03:53:21,095 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C38593%2C1701143599478, suffix=, logDir=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/MasterData/WALs/jenkins-hbase4.apache.org,38593,1701143599478, archiveDir=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/MasterData/oldWALs, maxLogs=10 2023-11-28 03:53:21,151 DEBUG [RS-EventLoopGroup-3-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40385,DS-3e272f40-ba73-4e34-81ee-8c3da9d3c697,DISK] 2023-11-28 03:53:21,159 DEBUG [RS-EventLoopGroup-3-2] asyncfs.ProtobufDecoder(123): Hadoop 3.2 and below use unshaded protobuf. java.lang.ClassNotFoundException: org.apache.hadoop.thirdparty.protobuf.MessageLite at java.net.URLClassLoader.findClass(URLClassLoader.java:387) at java.lang.ClassLoader.loadClass(ClassLoader.java:418) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352) at java.lang.ClassLoader.loadClass(ClassLoader.java:351) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.io.asyncfs.ProtobufDecoder.(ProtobufDecoder.java:118) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.processWriteBlockResponse(FanOutOneBlockAsyncDFSOutputHelper.java:340) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$100(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$4.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:424) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:557) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.addListener(DefaultPromise.java:185) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.initialize(FanOutOneBlockAsyncDFSOutputHelper.java:418) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$300(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:476) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:471) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setSuccess0(DefaultPromise.java:625) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:105) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:84) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:653) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:691) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:750) 2023-11-28 03:53:21,221 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/MasterData/WALs/jenkins-hbase4.apache.org,38593,1701143599478/jenkins-hbase4.apache.org%2C38593%2C1701143599478.1701143601104 2023-11-28 03:53:21,221 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:40385,DS-3e272f40-ba73-4e34-81ee-8c3da9d3c697,DISK]] 2023-11-28 03:53:21,222 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7854): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-11-28 03:53:21,222 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-28 03:53:21,225 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7894): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-11-28 03:53:21,226 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7897): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-11-28 03:53:21,284 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-11-28 03:53:21,291 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-11-28 03:53:21,315 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-11-28 03:53:21,330 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-28 03:53:21,335 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-11-28 03:53:21,337 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-11-28 03:53:21,351 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1055): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-11-28 03:53:21,356 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-11-28 03:53:21,357 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1072): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60978449, jitterRate=-0.09135030210018158}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-11-28 03:53:21,358 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(965): Region open journal for 1595e783b53d99cd5eef43b6debb2682: 2023-11-28 03:53:21,359 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-11-28 03:53:21,377 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=5 2023-11-28 03:53:21,377 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(561): Starting 5 core workers (bigger of cpus/4 or 16) with max (burst) worker count=50 2023-11-28 03:53:21,379 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-11-28 03:53:21,381 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(581): Recovered RegionProcedureStore lease in 1 msec 2023-11-28 03:53:21,414 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(595): Loaded RegionProcedureStore in 32 msec 2023-11-28 03:53:21,414 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(96): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-11-28 03:53:21,439 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [] 2023-11-28 03:53:21,444 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 0 possibly 'live' servers, and 0 'splitting'. 2023-11-28 03:53:21,467 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-11-28 03:53:21,470 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-11-28 03:53:21,472 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/balancer 2023-11-28 03:53:21,477 INFO [master/jenkins-hbase4:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-11-28 03:53:21,481 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/normalizer 2023-11-28 03:53:21,485 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-11-28 03:53:21,487 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/split 2023-11-28 03:53:21,487 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/merge 2023-11-28 03:53:21,500 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/snapshot-cleanup 2023-11-28 03:53:21,505 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-11-28 03:53:21,505 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-11-28 03:53:21,505 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-11-28 03:53:21,506 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(744): Active/primary master=jenkins-hbase4.apache.org,38593,1701143599478, sessionid=0x100321f3a150000, setting cluster-up flag (Was=false) 2023-11-28 03:53:21,528 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-11-28 03:53:21,536 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/flush-table-proc/acquired, /hbase/flush-table-proc/reached, /hbase/flush-table-proc/abort 2023-11-28 03:53:21,537 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,38593,1701143599478 2023-11-28 03:53:21,543 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-11-28 03:53:21,550 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/online-snapshot/acquired, /hbase/online-snapshot/reached, /hbase/online-snapshot/abort 2023-11-28 03:53:21,551 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,38593,1701143599478 2023-11-28 03:53:21,554 WARN [master/jenkins-hbase4:0:becomeActiveMaster] snapshot.SnapshotManager(304): Couldn't delete working snapshot directory: hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/.hbase-snapshot/.tmp 2023-11-28 03:53:21,593 INFO [RS:0;jenkins-hbase4:41203] regionserver.HRegionServer(954): ClusterId : ce5c0691-8473-42df-8a7d-422da22a0638 2023-11-28 03:53:21,598 DEBUG [RS:0;jenkins-hbase4:41203] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-11-28 03:53:21,603 DEBUG [RS:0;jenkins-hbase4:41203] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-11-28 03:53:21,604 DEBUG [RS:0;jenkins-hbase4:41203] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-11-28 03:53:21,614 DEBUG [RS:0;jenkins-hbase4:41203] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-11-28 03:53:21,617 DEBUG [RS:0;jenkins-hbase4:41203] zookeeper.ReadOnlyZKClient(139): Connect 0x5a7203e7 to 127.0.0.1:55641 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-11-28 03:53:21,627 DEBUG [RS:0;jenkins-hbase4:41203] ipc.AbstractRpcClient(189): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@54b5f6d9, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-11-28 03:53:21,628 DEBUG [RS:0;jenkins-hbase4:41203] ipc.AbstractRpcClient(189): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@508a6897, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-11-28 03:53:21,663 DEBUG [RS:0;jenkins-hbase4:41203] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase4:41203 2023-11-28 03:53:21,669 INFO [RS:0;jenkins-hbase4:41203] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-11-28 03:53:21,669 INFO [RS:0;jenkins-hbase4:41203] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-11-28 03:53:21,669 DEBUG [RS:0;jenkins-hbase4:41203] regionserver.HRegionServer(1025): About to register with Master. 2023-11-28 03:53:21,673 INFO [RS:0;jenkins-hbase4:41203] regionserver.HRegionServer(2814): reportForDuty to master=jenkins-hbase4.apache.org,38593,1701143599478 with isa=jenkins-hbase4.apache.org/172.31.14.131:41203, startcode=1701143600653 2023-11-28 03:53:21,679 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1028): Stored pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT; InitMetaProcedure table=hbase:meta 2023-11-28 03:53:21,689 DEBUG [RS:0;jenkins-hbase4:41203] ipc.RpcConnection(122): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-11-28 03:53:21,690 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-11-28 03:53:21,690 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-11-28 03:53:21,690 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-11-28 03:53:21,690 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-11-28 03:53:21,690 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase4:0, corePoolSize=10, maxPoolSize=10 2023-11-28 03:53:21,690 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-11-28 03:53:21,691 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-11-28 03:53:21,691 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-11-28 03:53:21,692 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1701143631692 2023-11-28 03:53:21,694 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-11-28 03:53:21,700 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT, locked=true; InitMetaProcedure table=hbase:meta 2023-11-28 03:53:21,701 INFO [PEWorker-1] procedure.InitMetaProcedure(71): BOOTSTRAP: creating hbase:meta region 2023-11-28 03:53:21,706 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-11-28 03:53:21,707 INFO [PEWorker-1] util.FSTableDescriptors(128): Creating new hbase:meta table descriptor 'hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-11-28 03:53:21,714 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-11-28 03:53:21,715 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-11-28 03:53:21,715 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-11-28 03:53:21,715 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-11-28 03:53:21,719 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-11-28 03:53:21,722 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-11-28 03:53:21,724 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-11-28 03:53:21,724 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-11-28 03:53:21,727 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-11-28 03:53:21,727 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-11-28 03:53:21,729 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1701143601728,5,FailOnTimeoutGroup] 2023-11-28 03:53:21,731 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1701143601729,5,FailOnTimeoutGroup] 2023-11-28 03:53:21,731 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-11-28 03:53:21,731 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1461): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-11-28 03:53:21,733 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-11-28 03:53:21,733 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-11-28 03:53:21,744 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-11-28 03:53:21,746 INFO [PEWorker-1] util.FSTableDescriptors(135): Updated hbase:meta table descriptor to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-11-28 03:53:21,746 INFO [PEWorker-1] regionserver.HRegion(7675): creating {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb 2023-11-28 03:53:21,757 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:60475, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.0 (auth:SIMPLE), service=RegionServerStatusService 2023-11-28 03:53:21,760 DEBUG [PEWorker-1] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-28 03:53:21,767 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-11-28 03:53:21,769 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/hbase/meta/1588230740/info 2023-11-28 03:53:21,770 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=38593] master.ServerManager(388): Registering regionserver=jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:21,770 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-11-28 03:53:21,771 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-28 03:53:21,772 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-11-28 03:53:21,775 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/hbase/meta/1588230740/rep_barrier 2023-11-28 03:53:21,776 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-11-28 03:53:21,777 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-28 03:53:21,777 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-11-28 03:53:21,779 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/hbase/meta/1588230740/table 2023-11-28 03:53:21,780 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-11-28 03:53:21,781 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-28 03:53:21,783 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/hbase/meta/1588230740 2023-11-28 03:53:21,784 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/hbase/meta/1588230740 2023-11-28 03:53:21,788 DEBUG [RS:0;jenkins-hbase4:41203] regionserver.HRegionServer(1598): Config from master: hbase.rootdir=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb 2023-11-28 03:53:21,788 DEBUG [RS:0;jenkins-hbase4:41203] regionserver.HRegionServer(1598): Config from master: fs.defaultFS=hdfs://localhost:34103 2023-11-28 03:53:21,788 DEBUG [RS:0;jenkins-hbase4:41203] regionserver.HRegionServer(1598): Config from master: hbase.master.info.port=-1 2023-11-28 03:53:21,788 DEBUG [PEWorker-1] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-11-28 03:53:21,791 DEBUG [PEWorker-1] regionserver.HRegion(1055): writing seq id for 1588230740 2023-11-28 03:53:21,794 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-11-28 03:53:21,797 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/hbase/meta/1588230740/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-11-28 03:53:21,797 DEBUG [RS:0;jenkins-hbase4:41203] zookeeper.ZKUtil(163): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:21,797 WARN [RS:0;jenkins-hbase4:41203] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-11-28 03:53:21,797 INFO [RS:0;jenkins-hbase4:41203] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-11-28 03:53:21,798 DEBUG [RS:0;jenkins-hbase4:41203] regionserver.HRegionServer(1951): logDir=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/WALs/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:21,798 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,41203,1701143600653] 2023-11-28 03:53:21,798 INFO [PEWorker-1] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61715717, jitterRate=-0.0803641527891159}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-11-28 03:53:21,798 DEBUG [PEWorker-1] regionserver.HRegion(965): Region open journal for 1588230740: 2023-11-28 03:53:21,799 DEBUG [PEWorker-1] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-11-28 03:53:21,799 INFO [PEWorker-1] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-11-28 03:53:21,799 DEBUG [PEWorker-1] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-11-28 03:53:21,799 DEBUG [PEWorker-1] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-11-28 03:53:21,799 DEBUG [PEWorker-1] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-11-28 03:53:21,805 INFO [PEWorker-1] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-11-28 03:53:21,805 DEBUG [PEWorker-1] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-11-28 03:53:21,812 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_ASSIGN_META, locked=true; InitMetaProcedure table=hbase:meta 2023-11-28 03:53:21,812 INFO [PEWorker-1] procedure.InitMetaProcedure(103): Going to assign meta 2023-11-28 03:53:21,813 DEBUG [RS:0;jenkins-hbase4:41203] zookeeper.ZKUtil(163): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:21,824 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-11-28 03:53:21,824 DEBUG [RS:0;jenkins-hbase4:41203] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-11-28 03:53:21,833 INFO [RS:0;jenkins-hbase4:41203] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-11-28 03:53:21,838 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-11-28 03:53:21,840 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-11-28 03:53:21,860 INFO [RS:0;jenkins-hbase4:41203] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-11-28 03:53:21,864 INFO [RS:0;jenkins-hbase4:41203] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-11-28 03:53:21,864 INFO [RS:0;jenkins-hbase4:41203] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-11-28 03:53:21,865 INFO [RS:0;jenkins-hbase4:41203] regionserver.HRegionServer$CompactionChecker(1840): CompactionChecker runs every PT1S 2023-11-28 03:53:21,872 INFO [RS:0;jenkins-hbase4:41203] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=100, unit=MILLISECONDS is enabled. 2023-11-28 03:53:21,872 DEBUG [RS:0;jenkins-hbase4:41203] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-11-28 03:53:21,873 DEBUG [RS:0;jenkins-hbase4:41203] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-11-28 03:53:21,873 DEBUG [RS:0;jenkins-hbase4:41203] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-11-28 03:53:21,873 DEBUG [RS:0;jenkins-hbase4:41203] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-11-28 03:53:21,873 DEBUG [RS:0;jenkins-hbase4:41203] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-11-28 03:53:21,873 DEBUG [RS:0;jenkins-hbase4:41203] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-11-28 03:53:21,873 DEBUG [RS:0;jenkins-hbase4:41203] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-11-28 03:53:21,873 DEBUG [RS:0;jenkins-hbase4:41203] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-11-28 03:53:21,873 DEBUG [RS:0;jenkins-hbase4:41203] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-11-28 03:53:21,874 DEBUG [RS:0;jenkins-hbase4:41203] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-11-28 03:53:21,874 INFO [RS:0;jenkins-hbase4:41203] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-11-28 03:53:21,874 INFO [RS:0;jenkins-hbase4:41203] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-11-28 03:53:21,875 INFO [RS:0;jenkins-hbase4:41203] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-11-28 03:53:21,890 INFO [RS:0;jenkins-hbase4:41203] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-11-28 03:53:21,892 INFO [RS:0;jenkins-hbase4:41203] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,41203,1701143600653-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-11-28 03:53:21,908 INFO [RS:0;jenkins-hbase4:41203] regionserver.Replication(203): jenkins-hbase4.apache.org,41203,1701143600653 started 2023-11-28 03:53:21,908 INFO [RS:0;jenkins-hbase4:41203] regionserver.HRegionServer(1640): Serving as jenkins-hbase4.apache.org,41203,1701143600653, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:41203, sessionid=0x100321f3a150001 2023-11-28 03:53:21,908 DEBUG [RS:0;jenkins-hbase4:41203] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-11-28 03:53:21,908 DEBUG [RS:0;jenkins-hbase4:41203] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:21,908 DEBUG [RS:0;jenkins-hbase4:41203] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,41203,1701143600653' 2023-11-28 03:53:21,908 DEBUG [RS:0;jenkins-hbase4:41203] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:53:21,909 DEBUG [RS:0;jenkins-hbase4:41203] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:53:21,910 DEBUG [RS:0;jenkins-hbase4:41203] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-11-28 03:53:21,910 DEBUG [RS:0;jenkins-hbase4:41203] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-11-28 03:53:21,910 DEBUG [RS:0;jenkins-hbase4:41203] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:21,910 DEBUG [RS:0;jenkins-hbase4:41203] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,41203,1701143600653' 2023-11-28 03:53:21,910 DEBUG [RS:0;jenkins-hbase4:41203] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-11-28 03:53:21,910 DEBUG [RS:0;jenkins-hbase4:41203] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-11-28 03:53:21,911 DEBUG [RS:0;jenkins-hbase4:41203] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-11-28 03:53:21,911 INFO [RS:0;jenkins-hbase4:41203] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-11-28 03:53:21,911 INFO [RS:0;jenkins-hbase4:41203] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-11-28 03:53:21,992 DEBUG [jenkins-hbase4:38593] assignment.AssignmentManager(2186): Processing assignQueue; systemServersCount=1, allServersCount=1 2023-11-28 03:53:21,995 INFO [PEWorker-3] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,41203,1701143600653, state=OPENING 2023-11-28 03:53:22,006 DEBUG [PEWorker-3] zookeeper.MetaTableLocator(240): hbase:meta region location doesn't exist, create it 2023-11-28 03:53:22,009 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-11-28 03:53:22,010 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-11-28 03:53:22,014 INFO [PEWorker-3] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=3, ppid=2, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,41203,1701143600653}] 2023-11-28 03:53:22,022 INFO [RS:0;jenkins-hbase4:41203] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C41203%2C1701143600653, suffix=, logDir=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/WALs/jenkins-hbase4.apache.org,41203,1701143600653, archiveDir=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/oldWALs, maxLogs=32 2023-11-28 03:53:22,044 DEBUG [RS-EventLoopGroup-3-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40385,DS-3e272f40-ba73-4e34-81ee-8c3da9d3c697,DISK] 2023-11-28 03:53:22,047 INFO [RS:0;jenkins-hbase4:41203] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/WALs/jenkins-hbase4.apache.org,41203,1701143600653/jenkins-hbase4.apache.org%2C41203%2C1701143600653.1701143602025 2023-11-28 03:53:22,048 DEBUG [RS:0;jenkins-hbase4:41203] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:40385,DS-3e272f40-ba73-4e34-81ee-8c3da9d3c697,DISK]] 2023-11-28 03:53:22,203 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:22,206 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(122): Using SIMPLE authentication for service=AdminService, sasl=false 2023-11-28 03:53:22,209 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:40964, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-11-28 03:53:22,221 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:meta,,1.1588230740 2023-11-28 03:53:22,221 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-11-28 03:53:22,225 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C41203%2C1701143600653.meta, suffix=.meta, logDir=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/WALs/jenkins-hbase4.apache.org,41203,1701143600653, archiveDir=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/oldWALs, maxLogs=32 2023-11-28 03:53:22,241 DEBUG [RS-EventLoopGroup-3-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40385,DS-3e272f40-ba73-4e34-81ee-8c3da9d3c697,DISK] 2023-11-28 03:53:22,245 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/WALs/jenkins-hbase4.apache.org,41203,1701143600653/jenkins-hbase4.apache.org%2C41203%2C1701143600653.meta.1701143602226.meta 2023-11-28 03:53:22,246 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:40385,DS-3e272f40-ba73-4e34-81ee-8c3da9d3c697,DISK]] 2023-11-28 03:53:22,246 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-11-28 03:53:22,247 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-11-28 03:53:22,263 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8550): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-11-28 03:53:22,268 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-11-28 03:53:22,274 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-11-28 03:53:22,274 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-28 03:53:22,274 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1588230740 2023-11-28 03:53:22,274 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1588230740 2023-11-28 03:53:22,277 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-11-28 03:53:22,279 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/hbase/meta/1588230740/info 2023-11-28 03:53:22,279 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/hbase/meta/1588230740/info 2023-11-28 03:53:22,280 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-11-28 03:53:22,280 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-28 03:53:22,281 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-11-28 03:53:22,282 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/hbase/meta/1588230740/rep_barrier 2023-11-28 03:53:22,282 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/hbase/meta/1588230740/rep_barrier 2023-11-28 03:53:22,283 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-11-28 03:53:22,284 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-28 03:53:22,284 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-11-28 03:53:22,286 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/hbase/meta/1588230740/table 2023-11-28 03:53:22,286 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/hbase/meta/1588230740/table 2023-11-28 03:53:22,286 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-11-28 03:53:22,287 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-28 03:53:22,289 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/hbase/meta/1588230740 2023-11-28 03:53:22,292 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/hbase/meta/1588230740 2023-11-28 03:53:22,295 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-11-28 03:53:22,298 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1588230740 2023-11-28 03:53:22,299 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67871111, jitterRate=0.011358365416526794}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-11-28 03:53:22,300 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1588230740: 2023-11-28 03:53:22,309 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2339): Post open deploy tasks for hbase:meta,,1.1588230740, pid=3, masterSystemTime=1701143602196 2023-11-28 03:53:22,325 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2366): Finished post open deploy task for hbase:meta,,1.1588230740 2023-11-28 03:53:22,326 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:meta,,1.1588230740 2023-11-28 03:53:22,326 INFO [PEWorker-5] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,41203,1701143600653, state=OPEN 2023-11-28 03:53:22,332 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/meta-region-server 2023-11-28 03:53:22,332 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-11-28 03:53:22,336 INFO [PEWorker-5] procedure2.ProcedureExecutor(1825): Finished subprocedure pid=3, resume processing ppid=2 2023-11-28 03:53:22,336 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=3, ppid=2, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,41203,1701143600653 in 318 msec 2023-11-28 03:53:22,342 INFO [PEWorker-1] procedure2.ProcedureExecutor(1825): Finished subprocedure pid=2, resume processing ppid=1 2023-11-28 03:53:22,342 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=2, ppid=1, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 514 msec 2023-11-28 03:53:22,347 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta in 740 msec 2023-11-28 03:53:22,347 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(953): Wait for region servers to report in: status=null, state=RUNNING, startTime=1701143602347, completionTime=-1 2023-11-28 03:53:22,347 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(808): Finished waiting on RegionServer count=1; waited=0ms, expected min=1 server(s), max=1 server(s), master is running 2023-11-28 03:53:22,348 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1527): Joining cluster... 2023-11-28 03:53:22,412 DEBUG [hconnection-0x1ad0a343-shared-pool-0] ipc.RpcConnection(122): Using SIMPLE authentication for service=ClientService, sasl=false 2023-11-28 03:53:22,416 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:48782, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-11-28 03:53:22,434 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1539): Number of RegionServers=1 2023-11-28 03:53:22,434 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1701143662434 2023-11-28 03:53:22,435 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1701143722435 2023-11-28 03:53:22,435 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1546): Joined the cluster in 86 msec 2023-11-28 03:53:22,464 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,38593,1701143599478-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-11-28 03:53:22,465 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,38593,1701143599478-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-11-28 03:53:22,465 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,38593,1701143599478-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-11-28 03:53:22,467 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase4:38593, period=300000, unit=MILLISECONDS is enabled. 2023-11-28 03:53:22,468 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-11-28 03:53:22,473 DEBUG [master/jenkins-hbase4:0.Chore.1] janitor.CatalogJanitor(175): 2023-11-28 03:53:22,489 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.TableNamespaceManager(92): Namespace table not found. Creating... 2023-11-28 03:53:22,490 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2148): Client=null/null create 'hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-11-28 03:53:22,503 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1028): Stored pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=hbase:namespace 2023-11-28 03:53:22,506 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_PRE_OPERATION 2023-11-28 03:53:22,508 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-11-28 03:53:22,528 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/.tmp/data/hbase/namespace/8a1f7e59ad225e2d27a98fd6fb53a214 2023-11-28 03:53:22,531 DEBUG [HFileArchiver-1] backup.HFileArchiver(153): Directory hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/.tmp/data/hbase/namespace/8a1f7e59ad225e2d27a98fd6fb53a214 empty. 2023-11-28 03:53:22,532 DEBUG [HFileArchiver-1] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/.tmp/data/hbase/namespace/8a1f7e59ad225e2d27a98fd6fb53a214 2023-11-28 03:53:22,532 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived hbase:namespace regions 2023-11-28 03:53:22,956 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/.tmp/data/hbase/namespace/.tabledesc/.tableinfo.0000000001 2023-11-28 03:53:22,958 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(7675): creating {ENCODED => 8a1f7e59ad225e2d27a98fd6fb53a214, NAME => 'hbase:namespace,,1701143602490.8a1f7e59ad225e2d27a98fd6fb53a214.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/.tmp 2023-11-28 03:53:23,372 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1701143602490.8a1f7e59ad225e2d27a98fd6fb53a214.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-28 03:53:23,373 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1604): Closing 8a1f7e59ad225e2d27a98fd6fb53a214, disabling compactions & flushes 2023-11-28 03:53:23,373 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1701143602490.8a1f7e59ad225e2d27a98fd6fb53a214. 2023-11-28 03:53:23,373 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1701143602490.8a1f7e59ad225e2d27a98fd6fb53a214. 2023-11-28 03:53:23,373 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1701143602490.8a1f7e59ad225e2d27a98fd6fb53a214. after waiting 0 ms 2023-11-28 03:53:23,373 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1701143602490.8a1f7e59ad225e2d27a98fd6fb53a214. 2023-11-28 03:53:23,373 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1838): Closed hbase:namespace,,1701143602490.8a1f7e59ad225e2d27a98fd6fb53a214. 2023-11-28 03:53:23,373 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1558): Region close journal for 8a1f7e59ad225e2d27a98fd6fb53a214: 2023-11-28 03:53:23,381 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ADD_TO_META 2023-11-28 03:53:23,395 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"hbase:namespace,,1701143602490.8a1f7e59ad225e2d27a98fd6fb53a214.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1701143603384"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1701143603384"}]},"ts":"1701143603384"} 2023-11-28 03:53:23,420 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-11-28 03:53:23,422 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-11-28 03:53:23,427 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1701143603422"}]},"ts":"1701143603422"} 2023-11-28 03:53:23,430 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLING in hbase:meta 2023-11-28 03:53:23,438 INFO [PEWorker-3] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=8a1f7e59ad225e2d27a98fd6fb53a214, ASSIGN}] 2023-11-28 03:53:23,440 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=8a1f7e59ad225e2d27a98fd6fb53a214, ASSIGN 2023-11-28 03:53:23,441 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=8a1f7e59ad225e2d27a98fd6fb53a214, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,41203,1701143600653; forceNewPlan=false, retain=false 2023-11-28 03:53:23,593 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=8a1f7e59ad225e2d27a98fd6fb53a214, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:23,593 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1701143602490.8a1f7e59ad225e2d27a98fd6fb53a214.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1701143603592"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1701143603592"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1701143603592"}]},"ts":"1701143603592"} 2023-11-28 03:53:23,598 INFO [PEWorker-5] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=6, ppid=5, state=RUNNABLE; OpenRegionProcedure 8a1f7e59ad225e2d27a98fd6fb53a214, server=jenkins-hbase4.apache.org,41203,1701143600653}] 2023-11-28 03:53:23,751 DEBUG [RSProcedureDispatcher-pool-1] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:23,758 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:namespace,,1701143602490.8a1f7e59ad225e2d27a98fd6fb53a214. 2023-11-28 03:53:23,759 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 8a1f7e59ad225e2d27a98fd6fb53a214, NAME => 'hbase:namespace,,1701143602490.8a1f7e59ad225e2d27a98fd6fb53a214.', STARTKEY => '', ENDKEY => ''} 2023-11-28 03:53:23,760 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace 8a1f7e59ad225e2d27a98fd6fb53a214 2023-11-28 03:53:23,760 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1701143602490.8a1f7e59ad225e2d27a98fd6fb53a214.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-28 03:53:23,760 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 8a1f7e59ad225e2d27a98fd6fb53a214 2023-11-28 03:53:23,760 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 8a1f7e59ad225e2d27a98fd6fb53a214 2023-11-28 03:53:23,762 INFO [StoreOpener-8a1f7e59ad225e2d27a98fd6fb53a214-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 8a1f7e59ad225e2d27a98fd6fb53a214 2023-11-28 03:53:23,765 DEBUG [StoreOpener-8a1f7e59ad225e2d27a98fd6fb53a214-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/hbase/namespace/8a1f7e59ad225e2d27a98fd6fb53a214/info 2023-11-28 03:53:23,765 DEBUG [StoreOpener-8a1f7e59ad225e2d27a98fd6fb53a214-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/hbase/namespace/8a1f7e59ad225e2d27a98fd6fb53a214/info 2023-11-28 03:53:23,765 INFO [StoreOpener-8a1f7e59ad225e2d27a98fd6fb53a214-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 8a1f7e59ad225e2d27a98fd6fb53a214 columnFamilyName info 2023-11-28 03:53:23,766 INFO [StoreOpener-8a1f7e59ad225e2d27a98fd6fb53a214-1] regionserver.HStore(310): Store=8a1f7e59ad225e2d27a98fd6fb53a214/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-28 03:53:23,767 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/hbase/namespace/8a1f7e59ad225e2d27a98fd6fb53a214 2023-11-28 03:53:23,768 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/hbase/namespace/8a1f7e59ad225e2d27a98fd6fb53a214 2023-11-28 03:53:23,774 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 8a1f7e59ad225e2d27a98fd6fb53a214 2023-11-28 03:53:23,777 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/hbase/namespace/8a1f7e59ad225e2d27a98fd6fb53a214/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-11-28 03:53:23,778 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 8a1f7e59ad225e2d27a98fd6fb53a214; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=65176186, jitterRate=-0.0287991464138031}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-11-28 03:53:23,778 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 8a1f7e59ad225e2d27a98fd6fb53a214: 2023-11-28 03:53:23,780 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2339): Post open deploy tasks for hbase:namespace,,1701143602490.8a1f7e59ad225e2d27a98fd6fb53a214., pid=6, masterSystemTime=1701143603751 2023-11-28 03:53:23,784 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2366): Finished post open deploy task for hbase:namespace,,1701143602490.8a1f7e59ad225e2d27a98fd6fb53a214. 2023-11-28 03:53:23,784 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:namespace,,1701143602490.8a1f7e59ad225e2d27a98fd6fb53a214. 2023-11-28 03:53:23,786 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=8a1f7e59ad225e2d27a98fd6fb53a214, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:23,786 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1701143602490.8a1f7e59ad225e2d27a98fd6fb53a214.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1701143603785"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1701143603785"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1701143603785"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1701143603785"}]},"ts":"1701143603785"} 2023-11-28 03:53:23,795 INFO [PEWorker-2] procedure2.ProcedureExecutor(1825): Finished subprocedure pid=6, resume processing ppid=5 2023-11-28 03:53:23,796 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=6, ppid=5, state=SUCCESS; OpenRegionProcedure 8a1f7e59ad225e2d27a98fd6fb53a214, server=jenkins-hbase4.apache.org,41203,1701143600653 in 193 msec 2023-11-28 03:53:23,799 INFO [PEWorker-3] procedure2.ProcedureExecutor(1825): Finished subprocedure pid=5, resume processing ppid=4 2023-11-28 03:53:23,799 INFO [PEWorker-3] procedure2.ProcedureExecutor(1409): Finished pid=5, ppid=4, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=8a1f7e59ad225e2d27a98fd6fb53a214, ASSIGN in 358 msec 2023-11-28 03:53:23,800 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-11-28 03:53:23,801 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1701143603801"}]},"ts":"1701143603801"} 2023-11-28 03:53:23,804 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLED in hbase:meta 2023-11-28 03:53:23,808 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_POST_OPERATION 2023-11-28 03:53:23,808 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/namespace 2023-11-28 03:53:23,810 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/namespace 2023-11-28 03:53:23,810 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-11-28 03:53:23,811 INFO [PEWorker-4] procedure2.ProcedureExecutor(1409): Finished pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace in 1.3160 sec 2023-11-28 03:53:23,855 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1028): Stored pid=7, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=default 2023-11-28 03:53:23,871 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-11-28 03:53:23,878 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default in 33 msec 2023-11-28 03:53:23,888 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1028): Stored pid=8, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=hbase 2023-11-28 03:53:23,899 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-11-28 03:53:23,904 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase in 15 msec 2023-11-28 03:53:23,917 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/default 2023-11-28 03:53:23,920 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/hbase 2023-11-28 03:53:23,921 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1083): Master has completed initialization 3.186sec 2023-11-28 03:53:23,923 INFO [master/jenkins-hbase4:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-11-28 03:53:23,925 INFO [master/jenkins-hbase4:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-11-28 03:53:23,925 INFO [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-11-28 03:53:23,925 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,38593,1701143599478-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-11-28 03:53:23,926 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,38593,1701143599478-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-11-28 03:53:23,936 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1175): Balancer post startup initialization complete, took 0 seconds 2023-11-28 03:53:24,001 DEBUG [Listener at localhost/42853] zookeeper.ReadOnlyZKClient(139): Connect 0x79a41f20 to 127.0.0.1:55641 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-11-28 03:53:24,008 DEBUG [Listener at localhost/42853] ipc.AbstractRpcClient(189): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@36d9c5a6, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-11-28 03:53:24,022 DEBUG [hconnection-0x64d3c1bf-shared-pool-0] ipc.RpcConnection(122): Using SIMPLE authentication for service=ClientService, sasl=false 2023-11-28 03:53:24,034 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:48790, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-11-28 03:53:24,044 INFO [Listener at localhost/42853] hbase.HBaseTestingUtility(1145): Minicluster is up; activeMaster=jenkins-hbase4.apache.org,38593,1701143599478 2023-11-28 03:53:24,046 DEBUG [Listener at localhost/42853] zookeeper.ReadOnlyZKClient(139): Connect 0x041eb7af to 127.0.0.1:55641 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-11-28 03:53:24,051 DEBUG [Listener at localhost/42853] ipc.AbstractRpcClient(189): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@65c25ebc, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-11-28 03:53:24,052 INFO [Listener at localhost/42853] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-11-28 03:53:24,100 INFO [Listener at localhost/42853] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthly Thread=214, OpenFileDescriptor=505, MaxFileDescriptor=60000, SystemLoadAverage=304, ProcessCount=172, AvailableMemoryMB=7173 2023-11-28 03:53:24,111 DEBUG [Listener at localhost/42853] ipc.RpcConnection(122): Using SIMPLE authentication for service=MasterService, sasl=false 2023-11-28 03:53:24,114 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:38360, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-11-28 03:53:24,126 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testMajorCompactionWithWeeklyFollowedByMonthly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-11-28 03:53:24,129 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] procedure2.ProcedureExecutor(1028): Stored pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:24,130 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_PRE_OPERATION 2023-11-28 03:53:24,133 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-11-28 03:53:24,135 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(700): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionWithWeeklyFollowedByMonthly" procId is: 9 2023-11-28 03:53:24,136 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9028932752f3f69b0ced4d93db937aaf 2023-11-28 03:53:24,137 DEBUG [HFileArchiver-2] backup.HFileArchiver(153): Directory hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9028932752f3f69b0ced4d93db937aaf empty. 2023-11-28 03:53:24,139 DEBUG [HFileArchiver-2] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9028932752f3f69b0ced4d93db937aaf 2023-11-28 03:53:24,139 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(328): Archived testMajorCompactionWithWeeklyFollowedByMonthly regions 2023-11-28 03:53:24,145 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-11-28 03:53:24,156 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/.tabledesc/.tableinfo.0000000001 2023-11-28 03:53:24,158 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(7675): creating {ENCODED => 9028932752f3f69b0ced4d93db937aaf, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthly,,1701143604123.9028932752f3f69b0ced4d93db937aaf.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMajorCompactionWithWeeklyFollowedByMonthly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}, regionDir=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/.tmp 2023-11-28 03:53:24,251 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-11-28 03:53:24,453 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-11-28 03:53:24,572 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthly,,1701143604123.9028932752f3f69b0ced4d93db937aaf.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-28 03:53:24,572 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1604): Closing 9028932752f3f69b0ced4d93db937aaf, disabling compactions & flushes 2023-11-28 03:53:24,572 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthly,,1701143604123.9028932752f3f69b0ced4d93db937aaf. 2023-11-28 03:53:24,572 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1701143604123.9028932752f3f69b0ced4d93db937aaf. 2023-11-28 03:53:24,573 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1701143604123.9028932752f3f69b0ced4d93db937aaf. after waiting 0 ms 2023-11-28 03:53:24,573 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthly,,1701143604123.9028932752f3f69b0ced4d93db937aaf. 2023-11-28 03:53:24,573 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthly,,1701143604123.9028932752f3f69b0ced4d93db937aaf. 2023-11-28 03:53:24,573 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1558): Region close journal for 9028932752f3f69b0ced4d93db937aaf: 2023-11-28 03:53:24,577 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_ADD_TO_META 2023-11-28 03:53:24,579 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1701143604123.9028932752f3f69b0ced4d93db937aaf.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1701143604579"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1701143604579"}]},"ts":"1701143604579"} 2023-11-28 03:53:24,581 INFO [PEWorker-2] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-11-28 03:53:24,583 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-11-28 03:53:24,583 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthly","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1701143604583"}]},"ts":"1701143604583"} 2023-11-28 03:53:24,585 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthly, state=ENABLING in hbase:meta 2023-11-28 03:53:24,590 INFO [PEWorker-2] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=9028932752f3f69b0ced4d93db937aaf, ASSIGN}] 2023-11-28 03:53:24,592 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=9028932752f3f69b0ced4d93db937aaf, ASSIGN 2023-11-28 03:53:24,594 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=9028932752f3f69b0ced4d93db937aaf, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,41203,1701143600653; forceNewPlan=false, retain=false 2023-11-28 03:53:24,745 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=9028932752f3f69b0ced4d93db937aaf, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:24,746 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1701143604123.9028932752f3f69b0ced4d93db937aaf.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1701143604745"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1701143604745"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1701143604745"}]},"ts":"1701143604745"} 2023-11-28 03:53:24,749 INFO [PEWorker-4] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=11, ppid=10, state=RUNNABLE; OpenRegionProcedure 9028932752f3f69b0ced4d93db937aaf, server=jenkins-hbase4.apache.org,41203,1701143600653}] 2023-11-28 03:53:24,755 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-11-28 03:53:24,903 DEBUG [RSProcedureDispatcher-pool-2] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:24,910 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testMajorCompactionWithWeeklyFollowedByMonthly,,1701143604123.9028932752f3f69b0ced4d93db937aaf. 2023-11-28 03:53:24,910 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 9028932752f3f69b0ced4d93db937aaf, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthly,,1701143604123.9028932752f3f69b0ced4d93db937aaf.', STARTKEY => '', ENDKEY => ''} 2023-11-28 03:53:24,910 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthly 9028932752f3f69b0ced4d93db937aaf 2023-11-28 03:53:24,910 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthly,,1701143604123.9028932752f3f69b0ced4d93db937aaf.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-28 03:53:24,910 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 9028932752f3f69b0ced4d93db937aaf 2023-11-28 03:53:24,910 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 9028932752f3f69b0ced4d93db937aaf 2023-11-28 03:53:24,913 INFO [StoreOpener-9028932752f3f69b0ced4d93db937aaf-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 9028932752f3f69b0ced4d93db937aaf 2023-11-28 03:53:24,916 DEBUG [StoreOpener-9028932752f3f69b0ced4d93db937aaf-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9028932752f3f69b0ced4d93db937aaf/family1 2023-11-28 03:53:24,916 DEBUG [StoreOpener-9028932752f3f69b0ced4d93db937aaf-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9028932752f3f69b0ced4d93db937aaf/family1 2023-11-28 03:53:24,919 INFO [StoreOpener-9028932752f3f69b0ced4d93db937aaf-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 9028932752f3f69b0ced4d93db937aaf columnFamilyName family1 2023-11-28 03:53:24,922 INFO [StoreOpener-9028932752f3f69b0ced4d93db937aaf-1] regionserver.HStore(310): Store=9028932752f3f69b0ced4d93db937aaf/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-28 03:53:24,924 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9028932752f3f69b0ced4d93db937aaf 2023-11-28 03:53:24,926 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9028932752f3f69b0ced4d93db937aaf 2023-11-28 03:53:24,931 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 9028932752f3f69b0ced4d93db937aaf 2023-11-28 03:53:24,935 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9028932752f3f69b0ced4d93db937aaf/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-11-28 03:53:24,936 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 9028932752f3f69b0ced4d93db937aaf; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63212714, jitterRate=-0.058057159185409546}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-11-28 03:53:24,936 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 9028932752f3f69b0ced4d93db937aaf: 2023-11-28 03:53:24,938 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2339): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthly,,1701143604123.9028932752f3f69b0ced4d93db937aaf., pid=11, masterSystemTime=1701143604903 2023-11-28 03:53:24,941 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2366): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthly,,1701143604123.9028932752f3f69b0ced4d93db937aaf. 2023-11-28 03:53:24,941 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testMajorCompactionWithWeeklyFollowedByMonthly,,1701143604123.9028932752f3f69b0ced4d93db937aaf. 2023-11-28 03:53:24,942 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=9028932752f3f69b0ced4d93db937aaf, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:24,942 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1701143604123.9028932752f3f69b0ced4d93db937aaf.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1701143604942"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1701143604942"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1701143604942"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1701143604942"}]},"ts":"1701143604942"} 2023-11-28 03:53:24,949 INFO [PEWorker-1] procedure2.ProcedureExecutor(1825): Finished subprocedure pid=11, resume processing ppid=10 2023-11-28 03:53:24,950 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=11, ppid=10, state=SUCCESS; OpenRegionProcedure 9028932752f3f69b0ced4d93db937aaf, server=jenkins-hbase4.apache.org,41203,1701143600653 in 197 msec 2023-11-28 03:53:24,953 INFO [PEWorker-2] procedure2.ProcedureExecutor(1825): Finished subprocedure pid=10, resume processing ppid=9 2023-11-28 03:53:24,953 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=10, ppid=9, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=9028932752f3f69b0ced4d93db937aaf, ASSIGN in 359 msec 2023-11-28 03:53:24,954 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-11-28 03:53:24,955 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthly","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1701143604954"}]},"ts":"1701143604954"} 2023-11-28 03:53:24,957 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthly, state=ENABLED in hbase:meta 2023-11-28 03:53:24,961 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_POST_OPERATION 2023-11-28 03:53:24,963 INFO [PEWorker-3] procedure2.ProcedureExecutor(1409): Finished pid=9, state=SUCCESS; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly in 834 msec 2023-11-28 03:53:25,258 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-11-28 03:53:25,258 INFO [Listener at localhost/42853] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionWithWeeklyFollowedByMonthly, procId: 9 completed 2023-11-28 03:53:25,282 DEBUG [hconnection-0x1ddb479a-metaLookup-shared--pool-0] ipc.RpcConnection(122): Using SIMPLE authentication for service=ClientService, sasl=false 2023-11-28 03:53:25,286 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:48792, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-11-28 03:53:25,421 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-28 03:53:25,427 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:25,428 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-11-28 03:53:25,428 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:53:25,428 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-11-28 03:53:25,428 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-11-28 03:53:25,429 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:25,429 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:25,431 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:53:25,431 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:25,431 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:53:25,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:53:25,432 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:25,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:25,432 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-28 03:53:25,432 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:25,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-28 03:53:25,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:25,433 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:25,435 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:25,436 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-11-28 03:53:25,436 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:53:25,437 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-11-28 03:53:25,438 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-28 03:53:25,438 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-28 03:53:25,438 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1701143604123.9028932752f3f69b0ced4d93db937aaf. 2023-11-28 03:53:25,438 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1701143604123.9028932752f3f69b0ced4d93db937aaf. started... 2023-11-28 03:53:25,439 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 9028932752f3f69b0ced4d93db937aaf 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-11-28 03:53:25,982 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015113004eb93389b61476ab8ffd92a37ddb1f1 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2015113004eb93389b61476ab8ffd92a37ddb1f1 2023-11-28 03:53:26,398 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9028932752f3f69b0ced4d93db937aaf/.tmp/family1/315a87470f334f31a04ab601cb06f059 2023-11-28 03:53:26,409 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9028932752f3f69b0ced4d93db937aaf/.tmp/family1/315a87470f334f31a04ab601cb06f059 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9028932752f3f69b0ced4d93db937aaf/family1/315a87470f334f31a04ab601cb06f059 2023-11-28 03:53:26,418 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9028932752f3f69b0ced4d93db937aaf/family1/315a87470f334f31a04ab601cb06f059, entries=1000, sequenceid=5, filesize=175.8 K 2023-11-28 03:53:26,422 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for 9028932752f3f69b0ced4d93db937aaf in 983ms, sequenceid=5, compaction requested=false 2023-11-28 03:53:26,424 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-11-28 03:53:26,426 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 9028932752f3f69b0ced4d93db937aaf: 2023-11-28 03:53:26,426 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1701143604123.9028932752f3f69b0ced4d93db937aaf. 2023-11-28 03:53:26,427 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-28 03:53:26,427 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-28 03:53:26,427 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:26,427 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-11-28 03:53:26,427 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-11-28 03:53:26,430 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:26,430 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:26,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:26,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:53:26,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:53:26,431 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:26,431 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-28 03:53:26,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:53:26,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:53:26,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:26,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:26,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:53:26,434 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-11-28 03:53:26,434 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-11-28 03:53:26,434 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4218a0a5[Count = 0] remaining members to acquire global barrier 2023-11-28 03:53:26,434 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:26,437 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:26,437 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:26,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:26,438 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-11-28 03:53:26,438 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-11-28 03:53:26,438 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,41203,1701143600653' in zk 2023-11-28 03:53:26,438 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:26,438 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-28 03:53:26,440 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-11-28 03:53:26,440 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:26,440 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:53:26,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:26,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:53:26,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:53:26,441 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-11-28 03:53:26,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:53:26,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:53:26,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:26,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:26,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:53:26,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:26,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:26,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,41203,1701143600653': 2023-11-28 03:53:26,446 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-11-28 03:53:26,446 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-28 03:53:26,446 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-11-28 03:53:26,446 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-28 03:53:26,446 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:26,446 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-28 03:53:26,449 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:26,449 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:26,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:26,449 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:26,449 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:53:26,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:26,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:53:26,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:53:26,450 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:53:26,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:53:26,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:53:26,450 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:26,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:26,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:26,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:53:26,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:26,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:26,452 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:26,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:53:26,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:26,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:26,461 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:26,461 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:53:26,461 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:26,461 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:26,461 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:26,461 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:53:26,461 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:53:26,461 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:26,461 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:53:26,462 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-11-28 03:53:26,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:53:26,462 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-28 03:53:26,461 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:26,462 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:26,462 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:53:26,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:53:26,464 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-11-28 03:53:26,464 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-28 03:53:26,564 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-28 03:53:26,569 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-28 03:53:26,580 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-28 03:53:26,583 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-11-28 03:53:26,583 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:26,583 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-11-28 03:53:26,583 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:53:26,584 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-11-28 03:53:26,584 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-11-28 03:53:26,585 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:26,585 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:26,595 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:26,595 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:53:26,595 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:53:26,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:53:26,595 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:26,595 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-28 03:53:26,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:26,597 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:26,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-28 03:53:26,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:26,598 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:26,598 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-11-28 03:53:26,598 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:26,598 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-11-28 03:53:26,598 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:53:26,599 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-11-28 03:53:26,599 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-28 03:53:26,599 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-28 03:53:26,599 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1701143604123.9028932752f3f69b0ced4d93db937aaf. 2023-11-28 03:53:26,599 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1701143604123.9028932752f3f69b0ced4d93db937aaf. started... 2023-11-28 03:53:26,599 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 9028932752f3f69b0ced4d93db937aaf 1/1 column families, dataSize=74 B heapSize=408 B 2023-11-28 03:53:27,034 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511285b6974886a4643fbba61e9ebbd80aa27 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201511285b6974886a4643fbba61e9ebbd80aa27 2023-11-28 03:53:27,446 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9028932752f3f69b0ced4d93db937aaf/.tmp/family1/23100322abfb46bb8976b7d3139e44c2 2023-11-28 03:53:27,457 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9028932752f3f69b0ced4d93db937aaf/.tmp/family1/23100322abfb46bb8976b7d3139e44c2 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9028932752f3f69b0ced4d93db937aaf/family1/23100322abfb46bb8976b7d3139e44c2 2023-11-28 03:53:27,467 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9028932752f3f69b0ced4d93db937aaf/family1/23100322abfb46bb8976b7d3139e44c2, entries=1, sequenceid=9, filesize=5.0 K 2023-11-28 03:53:27,468 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for 9028932752f3f69b0ced4d93db937aaf in 869ms, sequenceid=9, compaction requested=false 2023-11-28 03:53:27,469 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 9028932752f3f69b0ced4d93db937aaf: 2023-11-28 03:53:27,469 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1701143604123.9028932752f3f69b0ced4d93db937aaf. 2023-11-28 03:53:27,469 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-28 03:53:27,469 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-28 03:53:27,469 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:27,469 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-11-28 03:53:27,469 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-11-28 03:53:27,472 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:27,472 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:27,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:27,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:53:27,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:53:27,472 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:27,472 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-28 03:53:27,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:53:27,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:53:27,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:27,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:27,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:53:27,475 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-11-28 03:53:27,475 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@43746977[Count = 0] remaining members to acquire global barrier 2023-11-28 03:53:27,475 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-11-28 03:53:27,475 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:27,476 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:27,476 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:27,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:27,477 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-11-28 03:53:27,477 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-11-28 03:53:27,477 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,41203,1701143600653' in zk 2023-11-28 03:53:27,477 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:27,477 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-28 03:53:27,479 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-11-28 03:53:27,479 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:27,479 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:53:27,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:27,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:53:27,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:53:27,479 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-11-28 03:53:27,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:53:27,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:53:27,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:27,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:27,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:53:27,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:27,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:27,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,41203,1701143600653': 2023-11-28 03:53:27,483 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-11-28 03:53:27,483 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-11-28 03:53:27,483 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-28 03:53:27,483 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-28 03:53:27,483 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:27,483 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-28 03:53:27,486 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:27,486 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:27,486 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:27,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:27,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:53:27,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:53:27,486 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:53:27,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:27,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:53:27,487 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:53:27,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:53:27,487 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:27,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:27,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:27,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:53:27,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:27,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:27,489 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:27,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:53:27,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:27,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:27,494 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:27,495 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:27,495 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:53:27,495 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:27,495 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:53:27,495 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:53:27,495 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:27,495 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:53:27,495 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-11-28 03:53:27,495 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-28 03:53:27,495 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:27,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:53:27,496 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:27,496 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:27,496 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-11-28 03:53:27,496 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-28 03:53:27,496 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:53:27,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:53:27,596 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-28 03:53:27,598 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-28 03:53:27,607 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-28 03:53:27,609 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-11-28 03:53:27,610 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:27,610 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-11-28 03:53:27,610 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:53:27,610 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-11-28 03:53:27,610 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-11-28 03:53:27,611 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:27,611 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:27,613 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:27,613 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:53:27,613 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:53:27,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:53:27,613 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:27,613 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-28 03:53:27,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:27,614 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:27,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-28 03:53:27,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:27,615 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:27,615 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-11-28 03:53:27,615 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:27,615 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-11-28 03:53:27,615 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:53:27,615 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-11-28 03:53:27,616 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-28 03:53:27,616 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-28 03:53:27,616 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1701143604123.9028932752f3f69b0ced4d93db937aaf. 2023-11-28 03:53:27,616 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1701143604123.9028932752f3f69b0ced4d93db937aaf. started... 2023-11-28 03:53:27,616 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 9028932752f3f69b0ced4d93db937aaf 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-28 03:53:27,800 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-11-28 03:53:27,862 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:meta' 2023-11-28 03:53:27,863 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:namespace' 2023-11-28 03:53:28,046 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015120137206b0daf9549d4a0a4a0e08db00749 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2015120137206b0daf9549d4a0a4a0e08db00749 2023-11-28 03:53:28,059 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9028932752f3f69b0ced4d93db937aaf/.tmp/family1/b4cdc3303fde47b4b6a0b56aeb026fcd 2023-11-28 03:53:28,068 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9028932752f3f69b0ced4d93db937aaf/.tmp/family1/b4cdc3303fde47b4b6a0b56aeb026fcd as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9028932752f3f69b0ced4d93db937aaf/family1/b4cdc3303fde47b4b6a0b56aeb026fcd 2023-11-28 03:53:28,076 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9028932752f3f69b0ced4d93db937aaf/family1/b4cdc3303fde47b4b6a0b56aeb026fcd, entries=1, sequenceid=13, filesize=5.0 K 2023-11-28 03:53:28,078 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 9028932752f3f69b0ced4d93db937aaf in 462ms, sequenceid=13, compaction requested=true 2023-11-28 03:53:28,078 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 9028932752f3f69b0ced4d93db937aaf: 2023-11-28 03:53:28,078 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1701143604123.9028932752f3f69b0ced4d93db937aaf. 2023-11-28 03:53:28,078 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-28 03:53:28,078 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-28 03:53:28,078 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:28,079 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-11-28 03:53:28,079 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-11-28 03:53:28,081 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:28,081 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:28,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:28,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:53:28,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:53:28,082 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:28,082 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-28 03:53:28,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:53:28,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:53:28,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:28,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:28,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:53:28,084 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-11-28 03:53:28,084 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5073dd43[Count = 0] remaining members to acquire global barrier 2023-11-28 03:53:28,084 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-11-28 03:53:28,084 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:28,087 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:28,087 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:28,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:28,087 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-11-28 03:53:28,087 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-11-28 03:53:28,087 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,41203,1701143600653' in zk 2023-11-28 03:53:28,088 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:28,088 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-28 03:53:28,090 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-11-28 03:53:28,090 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:28,090 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:53:28,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:28,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:53:28,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:53:28,090 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-11-28 03:53:28,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:53:28,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:53:28,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:28,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:28,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:53:28,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:28,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:28,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,41203,1701143600653': 2023-11-28 03:53:28,094 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-11-28 03:53:28,094 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-11-28 03:53:28,094 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-28 03:53:28,095 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-28 03:53:28,095 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:28,095 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-28 03:53:28,097 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:28,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:28,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:53:28,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:53:28,097 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:28,097 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:53:28,098 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:28,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:28,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:53:28,098 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:53:28,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:53:28,099 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:28,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:28,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:28,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:53:28,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:28,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:28,101 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:28,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:53:28,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:28,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:28,106 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:28,106 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:28,106 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:28,106 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:53:28,106 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:28,106 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:53:28,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:53:28,106 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:28,106 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:28,106 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-11-28 03:53:28,106 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-28 03:53:28,106 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:28,106 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(399): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-11-28 03:53:28,107 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:53:28,106 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:53:28,107 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:53:28,107 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-11-28 03:53:28,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:53:28,108 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-28 03:53:28,208 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-28 03:53:28,210 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-28 03:53:28,218 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-28 03:53:28,220 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-11-28 03:53:28,220 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:28,220 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-11-28 03:53:28,221 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:53:28,221 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-11-28 03:53:28,221 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-11-28 03:53:28,222 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:28,222 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:28,225 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:53:28,225 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:28,225 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:53:28,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:53:28,225 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:28,225 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-28 03:53:28,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:28,226 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:28,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-28 03:53:28,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:28,227 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:28,227 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-11-28 03:53:28,227 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:28,227 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-11-28 03:53:28,227 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:53:28,228 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-11-28 03:53:28,228 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-28 03:53:28,228 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-28 03:53:28,228 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1701143604123.9028932752f3f69b0ced4d93db937aaf. 2023-11-28 03:53:28,228 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1701143604123.9028932752f3f69b0ced4d93db937aaf. started... 2023-11-28 03:53:28,228 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 9028932752f3f69b0ced4d93db937aaf 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-28 03:53:28,654 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512050342952ed711489a99520a3779491342 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201512050342952ed711489a99520a3779491342 2023-11-28 03:53:29,066 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9028932752f3f69b0ced4d93db937aaf/.tmp/family1/a3d469d7e1a64df69cef52308ded4d0d 2023-11-28 03:53:29,076 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9028932752f3f69b0ced4d93db937aaf/.tmp/family1/a3d469d7e1a64df69cef52308ded4d0d as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9028932752f3f69b0ced4d93db937aaf/family1/a3d469d7e1a64df69cef52308ded4d0d 2023-11-28 03:53:29,086 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9028932752f3f69b0ced4d93db937aaf/family1/a3d469d7e1a64df69cef52308ded4d0d, entries=1, sequenceid=17, filesize=5.0 K 2023-11-28 03:53:29,087 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 9028932752f3f69b0ced4d93db937aaf in 859ms, sequenceid=17, compaction requested=true 2023-11-28 03:53:29,087 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 9028932752f3f69b0ced4d93db937aaf: 2023-11-28 03:53:29,087 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1701143604123.9028932752f3f69b0ced4d93db937aaf. 2023-11-28 03:53:29,087 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-28 03:53:29,088 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-28 03:53:29,088 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:29,088 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-11-28 03:53:29,088 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-11-28 03:53:29,090 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:29,091 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:29,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:29,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:53:29,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:53:29,091 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:29,091 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-28 03:53:29,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:53:29,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:53:29,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:29,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:29,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:53:29,093 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-11-28 03:53:29,093 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@aaecb67[Count = 0] remaining members to acquire global barrier 2023-11-28 03:53:29,093 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-11-28 03:53:29,094 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:29,095 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:29,095 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:29,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:29,095 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-11-28 03:53:29,095 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-11-28 03:53:29,095 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,41203,1701143600653' in zk 2023-11-28 03:53:29,095 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:29,095 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-28 03:53:29,097 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:29,097 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-11-28 03:53:29,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:29,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:53:29,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:53:29,097 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:53:29,098 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-11-28 03:53:29,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:53:29,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:53:29,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:29,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:29,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:53:29,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:29,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:29,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,41203,1701143600653': 2023-11-28 03:53:29,101 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-11-28 03:53:29,101 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-11-28 03:53:29,101 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-28 03:53:29,101 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-28 03:53:29,101 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:29,101 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-28 03:53:29,108 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:29,108 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:29,108 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:29,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:29,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:53:29,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:53:29,108 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:53:29,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:29,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:53:29,109 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:53:29,109 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:29,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:53:29,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:29,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:29,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:53:29,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:29,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:29,111 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:29,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:53:29,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:29,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:29,115 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:29,115 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:29,115 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:29,115 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:29,115 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:29,115 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:29,115 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:53:29,115 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:53:29,115 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-11-28 03:53:29,116 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-28 03:53:29,115 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:29,116 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:53:29,116 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:53:29,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:53:29,116 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-11-28 03:53:29,116 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:53:29,116 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-28 03:53:29,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:53:29,217 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-28 03:53:29,218 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-28 03:53:29,226 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-28 03:53:29,229 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-11-28 03:53:29,229 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:29,229 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-11-28 03:53:29,229 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:53:29,230 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-11-28 03:53:29,230 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-11-28 03:53:29,230 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:29,230 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:29,232 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:53:29,232 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:29,232 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:53:29,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:53:29,233 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:29,233 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-28 03:53:29,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:29,235 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:29,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-28 03:53:29,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:29,235 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:29,235 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-11-28 03:53:29,235 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:29,236 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-11-28 03:53:29,236 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:53:29,236 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-11-28 03:53:29,236 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-28 03:53:29,236 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-28 03:53:29,236 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1701143604123.9028932752f3f69b0ced4d93db937aaf. 2023-11-28 03:53:29,236 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1701143604123.9028932752f3f69b0ced4d93db937aaf. started... 2023-11-28 03:53:29,237 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 9028932752f3f69b0ced4d93db937aaf 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-28 03:53:29,634 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-11-28 03:53:29,634 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint Metrics about HBase RegionObservers 2023-11-28 03:53:29,660 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151228eafdc379fee341e9a329a89146f7f787 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151228eafdc379fee341e9a329a89146f7f787 2023-11-28 03:53:30,072 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9028932752f3f69b0ced4d93db937aaf/.tmp/family1/b9c34e6a631547d2860002879550edde 2023-11-28 03:53:30,082 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9028932752f3f69b0ced4d93db937aaf/.tmp/family1/b9c34e6a631547d2860002879550edde as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9028932752f3f69b0ced4d93db937aaf/family1/b9c34e6a631547d2860002879550edde 2023-11-28 03:53:30,091 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9028932752f3f69b0ced4d93db937aaf/family1/b9c34e6a631547d2860002879550edde, entries=1, sequenceid=21, filesize=5.0 K 2023-11-28 03:53:30,092 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 9028932752f3f69b0ced4d93db937aaf in 855ms, sequenceid=21, compaction requested=true 2023-11-28 03:53:30,092 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 9028932752f3f69b0ced4d93db937aaf: 2023-11-28 03:53:30,092 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1701143604123.9028932752f3f69b0ced4d93db937aaf. 2023-11-28 03:53:30,092 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-28 03:53:30,093 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-28 03:53:30,093 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:30,094 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-11-28 03:53:30,094 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-11-28 03:53:30,096 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:30,096 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:30,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:30,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:53:30,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:53:30,097 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:30,097 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-28 03:53:30,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:53:30,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:53:30,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:30,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:30,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:53:30,100 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-11-28 03:53:30,100 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@d344f1f[Count = 0] remaining members to acquire global barrier 2023-11-28 03:53:30,100 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-11-28 03:53:30,100 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:30,102 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:30,102 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:30,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:30,102 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-11-28 03:53:30,102 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-11-28 03:53:30,102 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,41203,1701143600653' in zk 2023-11-28 03:53:30,102 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:30,102 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-28 03:53:30,104 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-11-28 03:53:30,104 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:30,104 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:53:30,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:30,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:53:30,104 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-11-28 03:53:30,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:53:30,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:53:30,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:53:30,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:30,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:30,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:53:30,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:30,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:30,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,41203,1701143600653': 2023-11-28 03:53:30,108 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-11-28 03:53:30,108 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-11-28 03:53:30,108 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-28 03:53:30,108 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-28 03:53:30,108 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:30,108 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-28 03:53:30,111 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:30,111 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:30,111 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:30,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:30,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:53:30,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:53:30,111 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:53:30,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:30,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:53:30,112 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:53:30,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:53:30,112 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:30,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:30,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:30,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:53:30,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:30,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:30,114 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:30,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:53:30,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:30,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:30,117 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:30,117 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:30,117 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:30,117 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:53:30,117 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:53:30,118 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:53:30,117 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:30,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:53:30,118 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-11-28 03:53:30,118 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-28 03:53:30,118 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:53:30,118 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:30,118 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:30,119 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:30,119 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:53:30,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:53:30,119 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-11-28 03:53:30,119 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-28 03:53:30,219 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-28 03:53:30,221 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-28 03:53:30,228 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-28 03:53:30,231 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-11-28 03:53:30,231 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:30,231 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-11-28 03:53:30,231 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:53:30,232 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-11-28 03:53:30,232 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-11-28 03:53:30,232 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:30,232 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:30,234 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:30,234 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:53:30,234 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:53:30,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:53:30,234 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:30,234 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-28 03:53:30,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:30,235 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:30,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-28 03:53:30,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:30,235 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:30,236 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-11-28 03:53:30,236 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:30,236 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-11-28 03:53:30,236 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:53:30,236 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-11-28 03:53:30,236 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-28 03:53:30,236 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-28 03:53:30,237 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1701143604123.9028932752f3f69b0ced4d93db937aaf. 2023-11-28 03:53:30,237 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1701143604123.9028932752f3f69b0ced4d93db937aaf. started... 2023-11-28 03:53:30,237 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 9028932752f3f69b0ced4d93db937aaf 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-28 03:53:30,257 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151231d7eaadd04c78408389e3adbe41d619aa to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151231d7eaadd04c78408389e3adbe41d619aa 2023-11-28 03:53:30,667 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9028932752f3f69b0ced4d93db937aaf/.tmp/family1/edc1db764e8842568b06977103be678c 2023-11-28 03:53:30,676 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9028932752f3f69b0ced4d93db937aaf/.tmp/family1/edc1db764e8842568b06977103be678c as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9028932752f3f69b0ced4d93db937aaf/family1/edc1db764e8842568b06977103be678c 2023-11-28 03:53:30,685 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9028932752f3f69b0ced4d93db937aaf/family1/edc1db764e8842568b06977103be678c, entries=1, sequenceid=25, filesize=5.0 K 2023-11-28 03:53:30,687 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 9028932752f3f69b0ced4d93db937aaf in 450ms, sequenceid=25, compaction requested=true 2023-11-28 03:53:30,687 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 9028932752f3f69b0ced4d93db937aaf: 2023-11-28 03:53:30,687 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1701143604123.9028932752f3f69b0ced4d93db937aaf. 2023-11-28 03:53:30,687 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-28 03:53:30,687 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-28 03:53:30,687 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:30,688 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-11-28 03:53:30,688 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-11-28 03:53:30,690 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:30,690 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:30,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:30,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:53:30,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:53:30,690 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:30,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:53:30,690 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-28 03:53:30,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:53:30,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:30,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:30,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:53:30,693 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-11-28 03:53:30,693 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6d6ba01f[Count = 0] remaining members to acquire global barrier 2023-11-28 03:53:30,693 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-11-28 03:53:30,693 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:30,695 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:30,695 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:30,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:30,695 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-11-28 03:53:30,695 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:30,695 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-28 03:53:30,695 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-11-28 03:53:30,695 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,41203,1701143600653' in zk 2023-11-28 03:53:30,697 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:30,697 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-11-28 03:53:30,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:30,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:53:30,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:53:30,697 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:53:30,698 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-11-28 03:53:30,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:53:30,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:53:30,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:30,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:30,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:53:30,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:30,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:30,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,41203,1701143600653': 2023-11-28 03:53:30,702 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-11-28 03:53:30,702 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-11-28 03:53:30,703 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-28 03:53:30,703 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-28 03:53:30,703 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:30,703 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-28 03:53:30,705 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:30,705 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:30,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:30,705 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:30,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:30,705 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:53:30,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:53:30,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:53:30,706 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:53:30,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:53:30,706 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:30,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:53:30,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:30,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:30,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:53:30,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:30,708 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:30,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:30,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:53:30,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:30,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:30,754 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:30,754 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:30,754 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:53:30,754 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:30,754 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:30,754 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:53:30,755 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(399): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-11-28 03:53:30,755 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-11-28 03:53:30,756 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-28 03:53:30,755 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:53:30,754 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:30,755 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:53:30,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:53:30,756 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:30,756 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:30,756 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-11-28 03:53:30,757 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-28 03:53:30,757 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:53:30,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:53:30,857 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-28 03:53:30,858 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-28 03:53:30,866 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-28 03:53:30,869 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-11-28 03:53:30,869 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:30,869 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-11-28 03:53:30,869 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:53:30,870 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-11-28 03:53:30,870 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-11-28 03:53:30,870 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:30,870 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:30,873 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:53:30,873 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:30,873 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:53:30,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:53:30,873 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:30,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:30,873 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-28 03:53:30,874 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:30,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-28 03:53:30,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:30,875 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:30,875 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:30,875 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-11-28 03:53:30,875 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:53:30,876 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-11-28 03:53:30,876 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-28 03:53:30,876 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-28 03:53:30,876 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1701143604123.9028932752f3f69b0ced4d93db937aaf. 2023-11-28 03:53:30,876 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1701143604123.9028932752f3f69b0ced4d93db937aaf. started... 2023-11-28 03:53:30,876 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 9028932752f3f69b0ced4d93db937aaf 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-28 03:53:30,900 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601014adf8c1976fa478193ec0d4f20bfb992 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201601014adf8c1976fa478193ec0d4f20bfb992 2023-11-28 03:53:30,908 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9028932752f3f69b0ced4d93db937aaf/.tmp/family1/0cdc3de7691945e79499113955f63224 2023-11-28 03:53:30,916 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9028932752f3f69b0ced4d93db937aaf/.tmp/family1/0cdc3de7691945e79499113955f63224 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9028932752f3f69b0ced4d93db937aaf/family1/0cdc3de7691945e79499113955f63224 2023-11-28 03:53:30,924 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9028932752f3f69b0ced4d93db937aaf/family1/0cdc3de7691945e79499113955f63224, entries=1, sequenceid=29, filesize=5.0 K 2023-11-28 03:53:30,925 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 9028932752f3f69b0ced4d93db937aaf in 49ms, sequenceid=29, compaction requested=true 2023-11-28 03:53:30,925 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 9028932752f3f69b0ced4d93db937aaf: 2023-11-28 03:53:30,926 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1701143604123.9028932752f3f69b0ced4d93db937aaf. 2023-11-28 03:53:30,926 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-28 03:53:30,926 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-28 03:53:30,926 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:30,926 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-11-28 03:53:30,926 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-11-28 03:53:30,931 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:30,931 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:30,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:30,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:53:30,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:53:30,932 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:30,932 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-28 03:53:30,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:53:30,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:53:30,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:30,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:30,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:53:30,934 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-11-28 03:53:30,934 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@23bcf134[Count = 0] remaining members to acquire global barrier 2023-11-28 03:53:30,934 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-11-28 03:53:30,934 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:30,936 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:30,936 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:30,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:30,936 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-11-28 03:53:30,936 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-11-28 03:53:30,936 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,41203,1701143600653' in zk 2023-11-28 03:53:30,936 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:30,936 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-28 03:53:30,938 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-11-28 03:53:30,938 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:30,938 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:53:30,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:30,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:53:30,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:53:30,939 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-11-28 03:53:30,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:53:30,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:53:30,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:30,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:30,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:53:30,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:30,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:30,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,41203,1701143600653': 2023-11-28 03:53:30,942 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-11-28 03:53:30,942 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-11-28 03:53:30,942 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-28 03:53:30,942 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-28 03:53:30,942 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:30,942 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-28 03:53:30,944 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:30,944 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:30,944 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:30,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:30,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:53:30,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:53:30,944 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:53:30,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:30,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:53:30,945 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:53:30,945 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:30,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:53:30,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:30,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:30,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:53:30,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:30,947 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:30,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:30,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:53:30,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:30,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:30,949 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:30,949 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:53:30,949 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:30,950 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:53:30,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:53:30,950 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-11-28 03:53:30,950 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(399): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-11-28 03:53:30,950 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:30,950 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-28 03:53:30,950 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:30,950 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:53:30,950 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:53:30,950 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:30,951 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:53:30,951 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:30,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:53:30,951 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:30,951 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-11-28 03:53:30,951 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-28 03:53:31,051 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-28 03:53:31,053 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-28 03:53:31,060 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-28 03:53:31,063 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-11-28 03:53:31,063 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:31,063 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-11-28 03:53:31,063 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:53:31,063 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-11-28 03:53:31,063 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-11-28 03:53:31,064 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:31,064 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:31,065 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:31,065 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:53:31,066 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:53:31,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:53:31,066 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:31,066 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-28 03:53:31,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:31,067 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:31,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-28 03:53:31,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:31,067 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:31,067 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-11-28 03:53:31,067 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:31,067 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-11-28 03:53:31,067 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:53:31,068 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-11-28 03:53:31,068 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-28 03:53:31,068 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-28 03:53:31,068 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1701143604123.9028932752f3f69b0ced4d93db937aaf. 2023-11-28 03:53:31,068 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1701143604123.9028932752f3f69b0ced4d93db937aaf. started... 2023-11-28 03:53:31,068 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 9028932752f3f69b0ced4d93db937aaf 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-28 03:53:31,490 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103374245d984cb4f67a2ee8f7f7b258d58 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20160103374245d984cb4f67a2ee8f7f7b258d58 2023-11-28 03:53:31,901 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9028932752f3f69b0ced4d93db937aaf/.tmp/family1/19380d35736143b7a416c018a90aac98 2023-11-28 03:53:31,910 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9028932752f3f69b0ced4d93db937aaf/.tmp/family1/19380d35736143b7a416c018a90aac98 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9028932752f3f69b0ced4d93db937aaf/family1/19380d35736143b7a416c018a90aac98 2023-11-28 03:53:31,917 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9028932752f3f69b0ced4d93db937aaf/family1/19380d35736143b7a416c018a90aac98, entries=1, sequenceid=33, filesize=5.0 K 2023-11-28 03:53:31,919 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 9028932752f3f69b0ced4d93db937aaf in 851ms, sequenceid=33, compaction requested=true 2023-11-28 03:53:31,919 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 9028932752f3f69b0ced4d93db937aaf: 2023-11-28 03:53:31,919 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1701143604123.9028932752f3f69b0ced4d93db937aaf. 2023-11-28 03:53:31,919 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-28 03:53:31,920 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-28 03:53:31,920 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:31,920 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-11-28 03:53:31,920 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-11-28 03:53:31,923 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:31,923 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:31,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:31,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:53:31,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:53:31,923 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:31,923 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-28 03:53:31,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:53:31,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:53:31,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:31,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:31,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:53:31,926 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-11-28 03:53:31,926 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5fb5f3d7[Count = 0] remaining members to acquire global barrier 2023-11-28 03:53:31,926 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-11-28 03:53:31,926 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:31,927 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:31,927 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:31,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:31,928 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-11-28 03:53:31,928 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-11-28 03:53:31,928 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,41203,1701143600653' in zk 2023-11-28 03:53:31,928 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:31,928 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-28 03:53:31,930 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-11-28 03:53:31,930 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:31,930 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:53:31,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:31,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:53:31,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:53:31,930 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-11-28 03:53:31,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:53:31,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:53:31,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:31,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:31,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:53:31,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:31,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:31,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,41203,1701143600653': 2023-11-28 03:53:31,933 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-11-28 03:53:31,933 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-11-28 03:53:31,934 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-28 03:53:31,934 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-28 03:53:31,934 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:31,934 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-28 03:53:31,935 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:31,935 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:31,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:31,936 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:31,936 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:53:31,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:31,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:53:31,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:53:31,936 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:31,936 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:53:31,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:53:31,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:53:31,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:31,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:31,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:53:31,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:31,938 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:31,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:31,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:53:31,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:31,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:31,942 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:53:31,942 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:53:31,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:53:31,942 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:31,942 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:53:31,942 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:31,943 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:31,943 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:53:31,943 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:31,943 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-11-28 03:53:31,943 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:53:31,943 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(399): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-11-28 03:53:31,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:53:31,943 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-28 03:53:31,943 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:31,944 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:31,944 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:31,944 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-11-28 03:53:31,944 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-28 03:53:32,044 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-28 03:53:32,046 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-28 03:53:32,053 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-28 03:53:32,056 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-11-28 03:53:32,057 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:32,057 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-11-28 03:53:32,058 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:53:32,058 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-11-28 03:53:32,058 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-11-28 03:53:32,058 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:32,059 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:32,060 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:32,060 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:53:32,060 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:53:32,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:53:32,061 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:32,061 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-28 03:53:32,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:32,061 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:32,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-28 03:53:32,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:32,062 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:32,062 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-11-28 03:53:32,062 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:32,062 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-11-28 03:53:32,062 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:53:32,062 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-11-28 03:53:32,062 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-28 03:53:32,063 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-28 03:53:32,063 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1701143604123.9028932752f3f69b0ced4d93db937aaf. 2023-11-28 03:53:32,063 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1701143604123.9028932752f3f69b0ced4d93db937aaf. started... 2023-11-28 03:53:32,063 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 9028932752f3f69b0ced4d93db937aaf 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-28 03:53:32,483 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20150907184e5f98ae8744319621eded3ee9fdd9 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20150907184e5f98ae8744319621eded3ee9fdd9 2023-11-28 03:53:32,893 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9028932752f3f69b0ced4d93db937aaf/.tmp/family1/d91f7c1a09794f94ab7acb1a545fd244 2023-11-28 03:53:32,903 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9028932752f3f69b0ced4d93db937aaf/.tmp/family1/d91f7c1a09794f94ab7acb1a545fd244 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9028932752f3f69b0ced4d93db937aaf/family1/d91f7c1a09794f94ab7acb1a545fd244 2023-11-28 03:53:32,910 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9028932752f3f69b0ced4d93db937aaf/family1/d91f7c1a09794f94ab7acb1a545fd244, entries=1, sequenceid=37, filesize=5.0 K 2023-11-28 03:53:32,912 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 9028932752f3f69b0ced4d93db937aaf in 848ms, sequenceid=37, compaction requested=true 2023-11-28 03:53:32,912 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 9028932752f3f69b0ced4d93db937aaf: 2023-11-28 03:53:32,912 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1701143604123.9028932752f3f69b0ced4d93db937aaf. 2023-11-28 03:53:32,912 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-28 03:53:32,912 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-28 03:53:32,912 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:32,913 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-11-28 03:53:32,913 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-11-28 03:53:32,915 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:32,915 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:32,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:32,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:53:32,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:53:32,915 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:32,915 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-28 03:53:32,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:53:32,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:53:32,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:32,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:32,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:53:32,917 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-11-28 03:53:32,917 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@70561389[Count = 0] remaining members to acquire global barrier 2023-11-28 03:53:32,917 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-11-28 03:53:32,917 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:32,919 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:32,919 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:32,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:32,919 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-11-28 03:53:32,919 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-11-28 03:53:32,919 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,41203,1701143600653' in zk 2023-11-28 03:53:32,919 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:32,919 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-28 03:53:32,922 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-11-28 03:53:32,922 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:32,922 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:53:32,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:32,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:53:32,922 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-11-28 03:53:32,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:53:32,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:53:32,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:53:32,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:32,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:32,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:53:32,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:32,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:32,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,41203,1701143600653': 2023-11-28 03:53:32,926 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-11-28 03:53:32,926 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-11-28 03:53:32,926 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-28 03:53:32,926 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-28 03:53:32,926 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:32,926 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-28 03:53:32,928 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:32,928 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:32,928 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:32,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:32,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:53:32,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:53:32,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:32,928 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:53:32,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:53:32,929 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:53:32,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:53:32,929 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:32,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:32,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:32,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:53:32,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:32,930 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:32,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:32,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:53:32,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:32,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:32,936 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:32,936 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:32,936 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:53:32,936 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:32,936 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:53:32,936 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:53:32,936 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-11-28 03:53:32,938 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(399): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-11-28 03:53:32,936 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:53:32,938 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-28 03:53:32,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:53:32,936 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:32,938 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:32,938 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:32,938 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:32,938 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-11-28 03:53:32,938 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:53:32,938 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-28 03:53:32,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:53:33,039 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-28 03:53:33,040 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-28 03:53:33,048 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-28 03:53:33,051 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-11-28 03:53:33,051 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:33,051 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-11-28 03:53:33,051 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:53:33,052 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-11-28 03:53:33,052 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-11-28 03:53:33,052 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:33,052 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:33,054 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:33,054 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:53:33,055 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:53:33,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:53:33,055 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:33,055 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-28 03:53:33,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:33,056 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:33,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-28 03:53:33,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:33,056 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:33,056 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-11-28 03:53:33,057 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:33,057 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-11-28 03:53:33,057 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:53:33,058 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-11-28 03:53:33,058 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-28 03:53:33,058 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-28 03:53:33,058 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1701143604123.9028932752f3f69b0ced4d93db937aaf. 2023-11-28 03:53:33,058 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1701143604123.9028932752f3f69b0ced4d93db937aaf. started... 2023-11-28 03:53:33,058 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 9028932752f3f69b0ced4d93db937aaf 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-28 03:53:33,479 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511201c99ca4147084f1cbf9398911c53ce21 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201511201c99ca4147084f1cbf9398911c53ce21 2023-11-28 03:53:33,889 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9028932752f3f69b0ced4d93db937aaf/.tmp/family1/6736715bf626460b82bae7e1ff6343b8 2023-11-28 03:53:33,897 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9028932752f3f69b0ced4d93db937aaf/.tmp/family1/6736715bf626460b82bae7e1ff6343b8 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9028932752f3f69b0ced4d93db937aaf/family1/6736715bf626460b82bae7e1ff6343b8 2023-11-28 03:53:33,904 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9028932752f3f69b0ced4d93db937aaf/family1/6736715bf626460b82bae7e1ff6343b8, entries=1, sequenceid=41, filesize=5.0 K 2023-11-28 03:53:33,905 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 9028932752f3f69b0ced4d93db937aaf in 847ms, sequenceid=41, compaction requested=true 2023-11-28 03:53:33,906 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 9028932752f3f69b0ced4d93db937aaf: 2023-11-28 03:53:33,906 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1701143604123.9028932752f3f69b0ced4d93db937aaf. 2023-11-28 03:53:33,906 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-28 03:53:33,906 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-28 03:53:33,906 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:33,906 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-11-28 03:53:33,906 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-11-28 03:53:33,908 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:33,908 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:33,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:33,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:53:33,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:53:33,909 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:33,909 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-28 03:53:33,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:53:33,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:53:33,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:33,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:33,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:53:33,910 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-11-28 03:53:33,910 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@400d2ec4[Count = 0] remaining members to acquire global barrier 2023-11-28 03:53:33,910 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-11-28 03:53:33,910 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:33,912 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:33,912 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:33,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:33,912 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-11-28 03:53:33,912 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-11-28 03:53:33,912 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:33,912 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-28 03:53:33,912 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,41203,1701143600653' in zk 2023-11-28 03:53:33,914 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-11-28 03:53:33,914 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:33,914 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:53:33,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:33,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:53:33,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:53:33,914 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-11-28 03:53:33,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:53:33,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:53:33,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:33,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:33,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:53:33,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:33,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:33,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,41203,1701143600653': 2023-11-28 03:53:33,917 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-11-28 03:53:33,917 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-11-28 03:53:33,918 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-28 03:53:33,918 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-28 03:53:33,918 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:33,918 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-28 03:53:33,921 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:33,921 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:33,921 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:33,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:33,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:53:33,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:53:33,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:33,921 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:53:33,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:53:33,922 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:53:33,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:53:33,922 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:33,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:33,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:33,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:53:33,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:33,923 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:33,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:33,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:53:33,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:33,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:33,926 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:33,926 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:33,926 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:33,926 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:33,926 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:33,926 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:33,926 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-11-28 03:53:33,926 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(399): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-11-28 03:53:33,926 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:53:33,927 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:53:33,927 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-28 03:53:33,927 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:53:33,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:53:33,926 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:33,927 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:53:33,928 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-11-28 03:53:33,928 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:53:33,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:53:33,928 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-28 03:53:34,028 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-28 03:53:34,030 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-28 03:53:34,034 DEBUG [Listener at localhost/42853] ipc.RpcConnection(122): Using SIMPLE authentication for service=AdminService, sasl=false 2023-11-28 03:53:34,037 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:54734, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-11-28 03:53:34,039 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=38593] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:34,053 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] procedure2.ProcedureExecutor(1028): Stored pid=12, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-11-28 03:53:34,054 DEBUG [PEWorker-4] locking.LockProcedure(309): LOCKED pid=12, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-11-28 03:53:34,056 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-11-28 03:53:34,056 INFO [PEWorker-4] procedure2.TimeoutExecutorThread(81): ADDED pid=12, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1701144214056 2023-11-28 03:53:34,057 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-11-28 03:53:34,062 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthly' and column 'family1' 2023-11-28 03:53:34,063 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-11-28 03:53:34,064 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] zookeeper.ReadOnlyZKClient(139): Connect 0x47660d3d to 127.0.0.1:55641 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-11-28 03:53:34,068 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] ipc.AbstractRpcClient(189): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@45e77de1, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-11-28 03:53:34,071 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-11-28 03:53:34,945 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103b61660f501c4438d85078118e569660d to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20160103b61660f501c4438d85078118e569660d 2023-11-28 03:53:34,946 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-11-28 03:53:34,974 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(122): Using SIMPLE authentication for service=ClientService, sasl=false 2023-11-28 03:53:34,976 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:41368, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-11-28 03:53:34,988 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(122): Using SIMPLE authentication for service=MasterService, sasl=false 2023-11-28 03:53:34,990 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:54750, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-11-28 03:53:35,031 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/51a7217f4d834b17b2c56cb19fdc6c38 first=Optional[r3] last=Optional[r6] 2023-11-28 03:53:35,044 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthly,,1701143604123.9028932752f3f69b0ced4d93db937aaf., hostname=jenkins-hbase4.apache.org,41203,1701143600653, seqNum=2 for row with hfile group [{family1,hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/51a7217f4d834b17b2c56cb19fdc6c38}] 2023-11-28 03:53:35,065 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HStore(763): Validating hfile at hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/51a7217f4d834b17b2c56cb19fdc6c38 for inclusion in 9028932752f3f69b0ced4d93db937aaf/family1 2023-11-28 03:53:35,072 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HStore(775): HFile bounds: first=r3 last=r6 2023-11-28 03:53:35,072 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HStore(777): Region bounds: first= last= 2023-11-28 03:53:35,072 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HRegion(2446): Flush status journal for 9028932752f3f69b0ced4d93db937aaf: 2023-11-28 03:53:35,074 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/51a7217f4d834b17b2c56cb19fdc6c38 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__o7q7rt39emp4gk42eqcsid3juuankbcl9n2sinpr4u915j1oss93eo6r43dkjvrg/family1/51a7217f4d834b17b2c56cb19fdc6c38 2023-11-28 03:53:35,077 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__o7q7rt39emp4gk42eqcsid3juuankbcl9n2sinpr4u915j1oss93eo6r43dkjvrg/family1/51a7217f4d834b17b2c56cb19fdc6c38 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9028932752f3f69b0ced4d93db937aaf/family1/4a2225b70764471da27279ff6b74981f_SeqId_44_ 2023-11-28 03:53:35,078 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HStore(849): Loaded HFile hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__o7q7rt39emp4gk42eqcsid3juuankbcl9n2sinpr4u915j1oss93eo6r43dkjvrg/family1/51a7217f4d834b17b2c56cb19fdc6c38 into 9028932752f3f69b0ced4d93db937aaf/family1 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9028932752f3f69b0ced4d93db937aaf/family1/4a2225b70764471da27279ff6b74981f_SeqId_44_ - updating store file list. 2023-11-28 03:53:35,085 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HStore(882): Loaded HFile hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9028932752f3f69b0ced4d93db937aaf/family1/4a2225b70764471da27279ff6b74981f_SeqId_44_ into 9028932752f3f69b0ced4d93db937aaf/family1 2023-11-28 03:53:35,085 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HStore(855): Successfully loaded hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__o7q7rt39emp4gk42eqcsid3juuankbcl9n2sinpr4u915j1oss93eo6r43dkjvrg/family1/51a7217f4d834b17b2c56cb19fdc6c38 into 9028932752f3f69b0ced4d93db937aaf/family1 (new location: hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9028932752f3f69b0ced4d93db937aaf/family1/4a2225b70764471da27279ff6b74981f_SeqId_44_) 2023-11-28 03:53:35,086 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__o7q7rt39emp4gk42eqcsid3juuankbcl9n2sinpr4u915j1oss93eo6r43dkjvrg/family1/51a7217f4d834b17b2c56cb19fdc6c38 2023-11-28 03:53:35,087 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-11-28 03:53:35,093 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 51357 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-11-28 03:53:35,095 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HStore(1912): 9028932752f3f69b0ced4d93db937aaf/family1 is initiating minor compaction 2023-11-28 03:53:35,096 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@26c4705f; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-28 03:53:35,097 INFO [RS:0;jenkins-hbase4:41203-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 9028932752f3f69b0ced4d93db937aaf/family1 in testMajorCompactionWithWeeklyFollowedByMonthly,,1701143604123.9028932752f3f69b0ced4d93db937aaf. 2023-11-28 03:53:35,097 INFO [RS:0;jenkins-hbase4:41203-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9028932752f3f69b0ced4d93db937aaf/family1/23100322abfb46bb8976b7d3139e44c2, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9028932752f3f69b0ced4d93db937aaf/family1/b4cdc3303fde47b4b6a0b56aeb026fcd, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9028932752f3f69b0ced4d93db937aaf/family1/a3d469d7e1a64df69cef52308ded4d0d, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9028932752f3f69b0ced4d93db937aaf/family1/b9c34e6a631547d2860002879550edde, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9028932752f3f69b0ced4d93db937aaf/family1/edc1db764e8842568b06977103be678c, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9028932752f3f69b0ced4d93db937aaf/family1/0cdc3de7691945e79499113955f63224, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9028932752f3f69b0ced4d93db937aaf/family1/19380d35736143b7a416c018a90aac98, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9028932752f3f69b0ced4d93db937aaf/family1/d91f7c1a09794f94ab7acb1a545fd244, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9028932752f3f69b0ced4d93db937aaf/family1/6736715bf626460b82bae7e1ff6343b8, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9028932752f3f69b0ced4d93db937aaf/family1/4a2225b70764471da27279ff6b74981f_SeqId_44_] into tmpdir=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9028932752f3f69b0ced4d93db937aaf/.tmp, totalSize=50.2 K 2023-11-28 03:53:35,098 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] compactions.Compactor(207): Compacting 23100322abfb46bb8976b7d3139e44c2, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-11-28 03:53:35,098 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] compactions.Compactor(207): Compacting b4cdc3303fde47b4b6a0b56aeb026fcd, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=13 2023-11-28 03:53:35,098 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] compactions.Compactor(207): Compacting a3d469d7e1a64df69cef52308ded4d0d, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=17 2023-11-28 03:53:35,098 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] compactions.Compactor(207): Compacting b9c34e6a631547d2860002879550edde, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=21 2023-11-28 03:53:35,098 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] compactions.Compactor(207): Compacting edc1db764e8842568b06977103be678c, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=25 2023-11-28 03:53:35,098 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] compactions.Compactor(207): Compacting 0cdc3de7691945e79499113955f63224, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=29 2023-11-28 03:53:35,098 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] compactions.Compactor(207): Compacting 19380d35736143b7a416c018a90aac98, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33 2023-11-28 03:53:35,099 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] compactions.Compactor(207): Compacting d91f7c1a09794f94ab7acb1a545fd244, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=37 2023-11-28 03:53:35,099 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] compactions.Compactor(207): Compacting 6736715bf626460b82bae7e1ff6343b8, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=41 2023-11-28 03:53:35,099 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] compactions.Compactor(207): Compacting 4a2225b70764471da27279ff6b74981f_SeqId_44_, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=45 2023-11-28 03:53:35,111 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-11-28 03:53:35,113 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-28 03:53:35,119 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151228eafdc379fee341e9a329a89146f7f787 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151228eafdc379fee341e9a329a89146f7f787 2023-11-28 03:53:35,121 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151231d7eaadd04c78408389e3adbe41d619aa to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151231d7eaadd04c78408389e3adbe41d619aa 2023-11-28 03:53:35,123 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201601014adf8c1976fa478193ec0d4f20bfb992 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201601014adf8c1976fa478193ec0d4f20bfb992 2023-11-28 03:53:35,125 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20160103374245d984cb4f67a2ee8f7f7b258d58 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20160103374245d984cb4f67a2ee8f7f7b258d58 2023-11-28 03:53:35,125 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthly 4 expired mob files are deleted 2023-11-28 03:53:35,125 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151228' 2023-11-28 03:53:35,126 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-11-28 03:53:35,126 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-11-28 03:53:35,126 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-11-28 03:53:35,126 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-11-28 03:53:35,126 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-11-28 03:53:35,126 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-11-28 03:53:35,127 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-11-28 03:53:35,127 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-11-28 03:53:35,164 INFO [RS:0;jenkins-hbase4:41203-shortCompactions-0] throttle.PressureAwareThroughputController(145): 9028932752f3f69b0ced4d93db937aaf#family1#compaction#10 average throughput is 0.21 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-11-28 03:53:35,201 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9028932752f3f69b0ced4d93db937aaf/.tmp/family1/b81e39510bbb48e9994c85c8f6763f08 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9028932752f3f69b0ced4d93db937aaf/family1/b81e39510bbb48e9994c85c8f6763f08 2023-11-28 03:53:35,234 INFO [RS:0;jenkins-hbase4:41203-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in 9028932752f3f69b0ced4d93db937aaf/family1 of 9028932752f3f69b0ced4d93db937aaf into b81e39510bbb48e9994c85c8f6763f08(size=6.6 K), total size for store is 182.5 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-11-28 03:53:35,234 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 9028932752f3f69b0ced4d93db937aaf: 2023-11-28 03:53:35,235 INFO [RS:0;jenkins-hbase4:41203-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthly,,1701143604123.9028932752f3f69b0ced4d93db937aaf., storeName=family1, fileCount=10, fileSize=50.2 K (5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.5 K), priority=2, time=1701143615094; duration=0sec 2023-11-28 03:53:35,236 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-28 03:53:35,284 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9028932752f3f69b0ced4d93db937aaf/family1/23100322abfb46bb8976b7d3139e44c2, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9028932752f3f69b0ced4d93db937aaf/family1/b4cdc3303fde47b4b6a0b56aeb026fcd, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9028932752f3f69b0ced4d93db937aaf/family1/a3d469d7e1a64df69cef52308ded4d0d, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9028932752f3f69b0ced4d93db937aaf/family1/b9c34e6a631547d2860002879550edde, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9028932752f3f69b0ced4d93db937aaf/family1/edc1db764e8842568b06977103be678c, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9028932752f3f69b0ced4d93db937aaf/family1/0cdc3de7691945e79499113955f63224, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9028932752f3f69b0ced4d93db937aaf/family1/19380d35736143b7a416c018a90aac98, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9028932752f3f69b0ced4d93db937aaf/family1/d91f7c1a09794f94ab7acb1a545fd244, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9028932752f3f69b0ced4d93db937aaf/family1/6736715bf626460b82bae7e1ff6343b8, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9028932752f3f69b0ced4d93db937aaf/family1/4a2225b70764471da27279ff6b74981f_SeqId_44_] to archive 2023-11-28 03:53:35,285 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-28 03:53:35,288 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9028932752f3f69b0ced4d93db937aaf/family1/23100322abfb46bb8976b7d3139e44c2 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9028932752f3f69b0ced4d93db937aaf/family1/23100322abfb46bb8976b7d3139e44c2 2023-11-28 03:53:35,290 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9028932752f3f69b0ced4d93db937aaf/family1/b4cdc3303fde47b4b6a0b56aeb026fcd to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9028932752f3f69b0ced4d93db937aaf/family1/b4cdc3303fde47b4b6a0b56aeb026fcd 2023-11-28 03:53:35,292 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9028932752f3f69b0ced4d93db937aaf/family1/a3d469d7e1a64df69cef52308ded4d0d to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9028932752f3f69b0ced4d93db937aaf/family1/a3d469d7e1a64df69cef52308ded4d0d 2023-11-28 03:53:35,293 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9028932752f3f69b0ced4d93db937aaf/family1/b9c34e6a631547d2860002879550edde to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9028932752f3f69b0ced4d93db937aaf/family1/b9c34e6a631547d2860002879550edde 2023-11-28 03:53:35,295 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9028932752f3f69b0ced4d93db937aaf/family1/edc1db764e8842568b06977103be678c to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9028932752f3f69b0ced4d93db937aaf/family1/edc1db764e8842568b06977103be678c 2023-11-28 03:53:35,297 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9028932752f3f69b0ced4d93db937aaf/family1/0cdc3de7691945e79499113955f63224 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9028932752f3f69b0ced4d93db937aaf/family1/0cdc3de7691945e79499113955f63224 2023-11-28 03:53:35,298 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9028932752f3f69b0ced4d93db937aaf/family1/19380d35736143b7a416c018a90aac98 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9028932752f3f69b0ced4d93db937aaf/family1/19380d35736143b7a416c018a90aac98 2023-11-28 03:53:35,300 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9028932752f3f69b0ced4d93db937aaf/family1/d91f7c1a09794f94ab7acb1a545fd244 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9028932752f3f69b0ced4d93db937aaf/family1/d91f7c1a09794f94ab7acb1a545fd244 2023-11-28 03:53:35,302 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9028932752f3f69b0ced4d93db937aaf/family1/6736715bf626460b82bae7e1ff6343b8 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9028932752f3f69b0ced4d93db937aaf/family1/6736715bf626460b82bae7e1ff6343b8 2023-11-28 03:53:35,304 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9028932752f3f69b0ced4d93db937aaf/family1/4a2225b70764471da27279ff6b74981f_SeqId_44_ to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9028932752f3f69b0ced4d93db937aaf/family1/4a2225b70764471da27279ff6b74981f_SeqId_44_ 2023-11-28 03:53:36,042 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512057968d2e34fe94319ac1b2e7c4ed9ccd8 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201512057968d2e34fe94319ac1b2e7c4ed9ccd8 2023-11-28 03:53:36,044 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-11-28 03:53:36,077 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/f28beba9ce1a46fd93f98f91178b05d8 first=Optional[r00] last=Optional[r2] 2023-11-28 03:53:36,078 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthly,,1701143604123.9028932752f3f69b0ced4d93db937aaf., hostname=jenkins-hbase4.apache.org,41203,1701143600653, seqNum=2 for row with hfile group [{family1,hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/f28beba9ce1a46fd93f98f91178b05d8}] 2023-11-28 03:53:36,081 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HStore(763): Validating hfile at hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/f28beba9ce1a46fd93f98f91178b05d8 for inclusion in 9028932752f3f69b0ced4d93db937aaf/family1 2023-11-28 03:53:36,085 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HStore(775): HFile bounds: first=r00 last=r2 2023-11-28 03:53:36,086 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HStore(777): Region bounds: first= last= 2023-11-28 03:53:36,086 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HRegion(2446): Flush status journal for 9028932752f3f69b0ced4d93db937aaf: 2023-11-28 03:53:36,087 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/f28beba9ce1a46fd93f98f91178b05d8 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__md4dgom5ld4g8i3s6l8vtjcggdhdthm1lovo5mg1b2ac6bui9ps5lhk56m6hioar/family1/f28beba9ce1a46fd93f98f91178b05d8 2023-11-28 03:53:36,090 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__md4dgom5ld4g8i3s6l8vtjcggdhdthm1lovo5mg1b2ac6bui9ps5lhk56m6hioar/family1/f28beba9ce1a46fd93f98f91178b05d8 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9028932752f3f69b0ced4d93db937aaf/family1/e79c4ea6d9394eafb10771cf21ffa421_SeqId_47_ 2023-11-28 03:53:36,091 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HStore(849): Loaded HFile hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__md4dgom5ld4g8i3s6l8vtjcggdhdthm1lovo5mg1b2ac6bui9ps5lhk56m6hioar/family1/f28beba9ce1a46fd93f98f91178b05d8 into 9028932752f3f69b0ced4d93db937aaf/family1 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9028932752f3f69b0ced4d93db937aaf/family1/e79c4ea6d9394eafb10771cf21ffa421_SeqId_47_ - updating store file list. 2023-11-28 03:53:36,098 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HStore(882): Loaded HFile hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9028932752f3f69b0ced4d93db937aaf/family1/e79c4ea6d9394eafb10771cf21ffa421_SeqId_47_ into 9028932752f3f69b0ced4d93db937aaf/family1 2023-11-28 03:53:36,099 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HStore(855): Successfully loaded hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__md4dgom5ld4g8i3s6l8vtjcggdhdthm1lovo5mg1b2ac6bui9ps5lhk56m6hioar/family1/f28beba9ce1a46fd93f98f91178b05d8 into 9028932752f3f69b0ced4d93db937aaf/family1 (new location: hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9028932752f3f69b0ced4d93db937aaf/family1/e79c4ea6d9394eafb10771cf21ffa421_SeqId_47_) 2023-11-28 03:53:36,099 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__md4dgom5ld4g8i3s6l8vtjcggdhdthm1lovo5mg1b2ac6bui9ps5lhk56m6hioar/family1/f28beba9ce1a46fd93f98f91178b05d8 2023-11-28 03:53:36,099 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-11-28 03:53:36,101 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 367249 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-11-28 03:53:36,101 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HStore(1912): 9028932752f3f69b0ced4d93db937aaf/family1 is initiating minor compaction (all files) 2023-11-28 03:53:36,101 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@4e9f4e5f; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-28 03:53:36,101 INFO [RS:0;jenkins-hbase4:41203-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 9028932752f3f69b0ced4d93db937aaf/family1 in testMajorCompactionWithWeeklyFollowedByMonthly,,1701143604123.9028932752f3f69b0ced4d93db937aaf. 2023-11-28 03:53:36,102 INFO [RS:0;jenkins-hbase4:41203-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9028932752f3f69b0ced4d93db937aaf/family1/315a87470f334f31a04ab601cb06f059, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9028932752f3f69b0ced4d93db937aaf/family1/b81e39510bbb48e9994c85c8f6763f08, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9028932752f3f69b0ced4d93db937aaf/family1/e79c4ea6d9394eafb10771cf21ffa421_SeqId_47_] into tmpdir=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9028932752f3f69b0ced4d93db937aaf/.tmp, totalSize=358.6 K 2023-11-28 03:53:36,102 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] compactions.Compactor(207): Compacting 315a87470f334f31a04ab601cb06f059, keycount=1000, bloomtype=ROW, size=175.8 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878824053 2023-11-28 03:53:36,103 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] compactions.Compactor(207): Compacting b81e39510bbb48e9994c85c8f6763f08, keycount=9, bloomtype=ROW, size=6.6 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621224053 2023-11-28 03:53:36,103 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] compactions.Compactor(207): Compacting e79c4ea6d9394eafb10771cf21ffa421_SeqId_47_, keycount=1002, bloomtype=ROW, size=176.2 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448878824053 2023-11-28 03:53:36,104 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-11-28 03:53:36,106 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-28 03:53:36,108 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2015113004eb93389b61476ab8ffd92a37ddb1f1 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2015113004eb93389b61476ab8ffd92a37ddb1f1 2023-11-28 03:53:36,110 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2015120137206b0daf9549d4a0a4a0e08db00749 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2015120137206b0daf9549d4a0a4a0e08db00749 2023-11-28 03:53:36,112 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201512050342952ed711489a99520a3779491342 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201512050342952ed711489a99520a3779491342 2023-11-28 03:53:36,112 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthly 3 expired mob files are deleted 2023-11-28 03:53:36,112 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151130' 2023-11-28 03:53:36,113 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-11-28 03:53:36,113 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] compactions.PartitionedMobCompactor(354): After compaction, there are 5 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-11-28 03:53:36,114 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-11-28 03:53:36,115 DEBUG [PEWorker-5] locking.LockProcedure(242): UNLOCKED pid=12, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-11-28 03:53:36,117 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=12, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE in 2.0710 sec 2023-11-28 03:53:36,143 INFO [RS:0;jenkins-hbase4:41203-shortCompactions-0] throttle.PressureAwareThroughputController(145): 9028932752f3f69b0ced4d93db937aaf#family1#compaction#11 average throughput is 6.61 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-11-28 03:53:36,287 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.HMaster$12(2454): Client=jenkins//172.31.14.131 modify table testMajorCompactionWithWeeklyFollowedByMonthly from 'testMajorCompactionWithWeeklyFollowedByMonthly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} to 'testMajorCompactionWithWeeklyFollowedByMonthly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} 2023-11-28 03:53:36,298 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] procedure2.ProcedureExecutor(1028): Stored pid=13, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:36,719 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/.tabledesc/.tableinfo.0000000002 2023-11-28 03:53:36,721 DEBUG [PEWorker-1] util.FSTableDescriptors(519): Deleted hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/.tabledesc/.tableinfo.0000000001 2023-11-28 03:53:36,721 INFO [PEWorker-1] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/.tabledesc/.tableinfo.0000000002 2023-11-28 03:53:36,726 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=14, ppid=13, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly}] 2023-11-28 03:53:36,740 INFO [PEWorker-2] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=15, ppid=14, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=9028932752f3f69b0ced4d93db937aaf, REOPEN/MOVE}] 2023-11-28 03:53:36,742 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=15, ppid=14, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=9028932752f3f69b0ced4d93db937aaf, REOPEN/MOVE 2023-11-28 03:53:36,743 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=9028932752f3f69b0ced4d93db937aaf, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:36,743 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1701143604123.9028932752f3f69b0ced4d93db937aaf.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1701143616742"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1701143616742"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1701143616742"}]},"ts":"1701143616742"} 2023-11-28 03:53:36,746 INFO [PEWorker-3] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=16, ppid=15, state=RUNNABLE; CloseRegionProcedure 9028932752f3f69b0ced4d93db937aaf, server=jenkins-hbase4.apache.org,41203,1701143600653}] 2023-11-28 03:53:36,902 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:36,905 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 9028932752f3f69b0ced4d93db937aaf 2023-11-28 03:53:36,905 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 9028932752f3f69b0ced4d93db937aaf, disabling compactions & flushes 2023-11-28 03:53:36,905 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1858): waiting for 1 compactions to complete for region testMajorCompactionWithWeeklyFollowedByMonthly,,1701143604123.9028932752f3f69b0ced4d93db937aaf. 2023-11-28 03:53:37,385 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9028932752f3f69b0ced4d93db937aaf/.tmp/family1/0da8253bfc4e4ceca379c9093c2b405c as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9028932752f3f69b0ced4d93db937aaf/family1/0da8253bfc4e4ceca379c9093c2b405c 2023-11-28 03:53:37,395 INFO [RS:0;jenkins-hbase4:41203-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 9028932752f3f69b0ced4d93db937aaf/family1 of 9028932752f3f69b0ced4d93db937aaf into 0da8253bfc4e4ceca379c9093c2b405c(size=177.2 K), total size for store is 177.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-11-28 03:53:37,395 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 9028932752f3f69b0ced4d93db937aaf: 2023-11-28 03:53:37,395 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthly,,1701143604123.9028932752f3f69b0ced4d93db937aaf. 2023-11-28 03:53:37,396 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1701143604123.9028932752f3f69b0ced4d93db937aaf. 2023-11-28 03:53:37,396 INFO [RS:0;jenkins-hbase4:41203-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthly,,1701143604123.9028932752f3f69b0ced4d93db937aaf., storeName=family1, fileCount=3, fileSize=358.6 K (175.8 K, 6.6 K, 176.2 K), priority=2, time=1701143616101; duration=1sec 2023-11-28 03:53:37,396 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1701143604123.9028932752f3f69b0ced4d93db937aaf. after waiting 0 ms 2023-11-28 03:53:37,396 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-28 03:53:37,396 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthly,,1701143604123.9028932752f3f69b0ced4d93db937aaf. 2023-11-28 03:53:37,399 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthly,,1701143604123.9028932752f3f69b0ced4d93db937aaf.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9028932752f3f69b0ced4d93db937aaf/family1/315a87470f334f31a04ab601cb06f059, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9028932752f3f69b0ced4d93db937aaf/family1/b81e39510bbb48e9994c85c8f6763f08, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9028932752f3f69b0ced4d93db937aaf/family1/e79c4ea6d9394eafb10771cf21ffa421_SeqId_47_] to archive 2023-11-28 03:53:37,400 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthly,,1701143604123.9028932752f3f69b0ced4d93db937aaf.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-11-28 03:53:37,402 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthly,,1701143604123.9028932752f3f69b0ced4d93db937aaf.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9028932752f3f69b0ced4d93db937aaf/family1/315a87470f334f31a04ab601cb06f059 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9028932752f3f69b0ced4d93db937aaf/family1/315a87470f334f31a04ab601cb06f059 2023-11-28 03:53:37,404 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthly,,1701143604123.9028932752f3f69b0ced4d93db937aaf.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9028932752f3f69b0ced4d93db937aaf/family1/b81e39510bbb48e9994c85c8f6763f08 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9028932752f3f69b0ced4d93db937aaf/family1/b81e39510bbb48e9994c85c8f6763f08 2023-11-28 03:53:37,406 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthly,,1701143604123.9028932752f3f69b0ced4d93db937aaf.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9028932752f3f69b0ced4d93db937aaf/family1/e79c4ea6d9394eafb10771cf21ffa421_SeqId_47_ to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9028932752f3f69b0ced4d93db937aaf/family1/e79c4ea6d9394eafb10771cf21ffa421_SeqId_47_ 2023-11-28 03:53:37,412 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9028932752f3f69b0ced4d93db937aaf/recovered.edits/50.seqid, newMaxSeqId=50, maxSeqId=1 2023-11-28 03:53:37,415 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthly,,1701143604123.9028932752f3f69b0ced4d93db937aaf. 2023-11-28 03:53:37,415 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 9028932752f3f69b0ced4d93db937aaf: 2023-11-28 03:53:37,415 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3512): Not adding moved region record: 9028932752f3f69b0ced4d93db937aaf to self. 2023-11-28 03:53:37,417 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 9028932752f3f69b0ced4d93db937aaf 2023-11-28 03:53:37,418 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=9028932752f3f69b0ced4d93db937aaf, regionState=CLOSED 2023-11-28 03:53:37,418 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1701143604123.9028932752f3f69b0ced4d93db937aaf.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1701143617418"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1701143617418"}]},"ts":"1701143617418"} 2023-11-28 03:53:37,422 INFO [PEWorker-5] procedure2.ProcedureExecutor(1825): Finished subprocedure pid=16, resume processing ppid=15 2023-11-28 03:53:37,422 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=16, ppid=15, state=SUCCESS; CloseRegionProcedure 9028932752f3f69b0ced4d93db937aaf, server=jenkins-hbase4.apache.org,41203,1701143600653 in 675 msec 2023-11-28 03:53:37,423 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=15, ppid=14, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=9028932752f3f69b0ced4d93db937aaf, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,41203,1701143600653; forceNewPlan=false, retain=true 2023-11-28 03:53:37,573 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=9028932752f3f69b0ced4d93db937aaf, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:37,574 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1701143604123.9028932752f3f69b0ced4d93db937aaf.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1701143617573"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1701143617573"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1701143617573"}]},"ts":"1701143617573"} 2023-11-28 03:53:37,577 INFO [PEWorker-2] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=17, ppid=15, state=RUNNABLE; OpenRegionProcedure 9028932752f3f69b0ced4d93db937aaf, server=jenkins-hbase4.apache.org,41203,1701143600653}] 2023-11-28 03:53:37,729 DEBUG [RSProcedureDispatcher-pool-1] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:37,733 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testMajorCompactionWithWeeklyFollowedByMonthly,,1701143604123.9028932752f3f69b0ced4d93db937aaf. 2023-11-28 03:53:37,733 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 9028932752f3f69b0ced4d93db937aaf, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthly,,1701143604123.9028932752f3f69b0ced4d93db937aaf.', STARTKEY => '', ENDKEY => ''} 2023-11-28 03:53:37,734 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthly 9028932752f3f69b0ced4d93db937aaf 2023-11-28 03:53:37,734 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthly,,1701143604123.9028932752f3f69b0ced4d93db937aaf.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-28 03:53:37,734 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 9028932752f3f69b0ced4d93db937aaf 2023-11-28 03:53:37,734 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 9028932752f3f69b0ced4d93db937aaf 2023-11-28 03:53:37,735 INFO [StoreOpener-9028932752f3f69b0ced4d93db937aaf-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 9028932752f3f69b0ced4d93db937aaf 2023-11-28 03:53:37,737 DEBUG [StoreOpener-9028932752f3f69b0ced4d93db937aaf-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9028932752f3f69b0ced4d93db937aaf/family1 2023-11-28 03:53:37,737 DEBUG [StoreOpener-9028932752f3f69b0ced4d93db937aaf-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9028932752f3f69b0ced4d93db937aaf/family1 2023-11-28 03:53:37,737 INFO [StoreOpener-9028932752f3f69b0ced4d93db937aaf-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 9028932752f3f69b0ced4d93db937aaf columnFamilyName family1 2023-11-28 03:53:37,745 DEBUG [StoreOpener-9028932752f3f69b0ced4d93db937aaf-1] regionserver.HStore(539): loaded hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9028932752f3f69b0ced4d93db937aaf/family1/0da8253bfc4e4ceca379c9093c2b405c 2023-11-28 03:53:37,745 INFO [StoreOpener-9028932752f3f69b0ced4d93db937aaf-1] regionserver.HStore(310): Store=9028932752f3f69b0ced4d93db937aaf/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-28 03:53:37,747 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9028932752f3f69b0ced4d93db937aaf 2023-11-28 03:53:37,748 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9028932752f3f69b0ced4d93db937aaf 2023-11-28 03:53:37,751 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 9028932752f3f69b0ced4d93db937aaf 2023-11-28 03:53:37,752 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 9028932752f3f69b0ced4d93db937aaf; next sequenceid=51; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=65444071, jitterRate=-0.02480734884738922}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-11-28 03:53:37,752 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 9028932752f3f69b0ced4d93db937aaf: 2023-11-28 03:53:37,754 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2339): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthly,,1701143604123.9028932752f3f69b0ced4d93db937aaf., pid=17, masterSystemTime=1701143617729 2023-11-28 03:53:37,755 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2366): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthly,,1701143604123.9028932752f3f69b0ced4d93db937aaf. 2023-11-28 03:53:37,755 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testMajorCompactionWithWeeklyFollowedByMonthly,,1701143604123.9028932752f3f69b0ced4d93db937aaf. 2023-11-28 03:53:37,756 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=9028932752f3f69b0ced4d93db937aaf, regionState=OPEN, openSeqNum=51, regionLocation=jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:37,756 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1701143604123.9028932752f3f69b0ced4d93db937aaf.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1701143617756"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1701143617756"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1701143617756"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1701143617756"}]},"ts":"1701143617756"} 2023-11-28 03:53:37,760 INFO [PEWorker-4] procedure2.ProcedureExecutor(1825): Finished subprocedure pid=17, resume processing ppid=15 2023-11-28 03:53:37,760 INFO [PEWorker-4] procedure2.ProcedureExecutor(1409): Finished pid=17, ppid=15, state=SUCCESS; OpenRegionProcedure 9028932752f3f69b0ced4d93db937aaf, server=jenkins-hbase4.apache.org,41203,1701143600653 in 181 msec 2023-11-28 03:53:37,763 INFO [PEWorker-5] procedure2.ProcedureExecutor(1825): Finished subprocedure pid=15, resume processing ppid=14 2023-11-28 03:53:37,763 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=15, ppid=14, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=9028932752f3f69b0ced4d93db937aaf, REOPEN/MOVE in 1.0200 sec 2023-11-28 03:53:37,766 INFO [PEWorker-1] procedure2.ProcedureExecutor(1825): Finished subprocedure pid=14, resume processing ppid=13 2023-11-28 03:53:37,766 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=14, ppid=13, state=SUCCESS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly in 1.0390 sec 2023-11-28 03:53:37,769 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=13, state=SUCCESS; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly in 1.4760 sec 2023-11-28 03:53:37,772 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(1230): Checking to see if procedure is done pid=13 2023-11-28 03:53:37,779 INFO [Listener at localhost/42853] compactions.TestMobCompactor(278): alter status finished 2023-11-28 03:53:37,780 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=38593] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-28 03:53:37,782 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] procedure2.ProcedureExecutor(1028): Stored pid=18, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-11-28 03:53:37,783 DEBUG [PEWorker-3] locking.LockProcedure(309): LOCKED pid=18, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-11-28 03:53:37,784 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-11-28 03:53:37,784 INFO [PEWorker-3] procedure2.TimeoutExecutorThread(81): ADDED pid=18, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1701144217784 2023-11-28 03:53:37,785 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-11-28 03:53:37,785 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 5 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthly' and column 'family1' 2023-11-28 03:53:37,785 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-11-28 03:53:37,787 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] zookeeper.ReadOnlyZKClient(139): Connect 0x51ad0549 to 127.0.0.1:55641 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-11-28 03:53:37,792 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] ipc.AbstractRpcClient(189): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@4d4bf748, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-11-28 03:53:37,792 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-11-28 03:53:37,792 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-11-28 03:53:37,792 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-11-28 03:53:37,793 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-11-28 03:53:37,793 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-11-28 03:53:38,631 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128ab2df29adb6643668f1a5e967c270338 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151128ab2df29adb6643668f1a5e967c270338 2023-11-28 03:53:38,632 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-11-28 03:53:38,646 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(122): Using SIMPLE authentication for service=ClientService, sasl=false 2023-11-28 03:53:38,647 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:41370, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-11-28 03:53:38,654 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(122): Using SIMPLE authentication for service=MasterService, sasl=false 2023-11-28 03:53:38,656 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:54760, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-11-28 03:53:38,669 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/c5ddc0541ce04cc5a1b0722b78375f45 first=Optional[r05] last=Optional[r8] 2023-11-28 03:53:38,670 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthly,,1701143604123.9028932752f3f69b0ced4d93db937aaf., hostname=jenkins-hbase4.apache.org,41203,1701143600653, seqNum=51 for row with hfile group [{family1,hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/c5ddc0541ce04cc5a1b0722b78375f45}] 2023-11-28 03:53:38,676 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HStore(763): Validating hfile at hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/c5ddc0541ce04cc5a1b0722b78375f45 for inclusion in 9028932752f3f69b0ced4d93db937aaf/family1 2023-11-28 03:53:38,680 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HStore(775): HFile bounds: first=r05 last=r8 2023-11-28 03:53:38,680 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HStore(777): Region bounds: first= last= 2023-11-28 03:53:38,681 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HRegion(2446): Flush status journal for 9028932752f3f69b0ced4d93db937aaf: 2023-11-28 03:53:38,682 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/c5ddc0541ce04cc5a1b0722b78375f45 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__rrgepe6ttujvudbhq99k3mvj7cai9j9fu249ie83lrhvtb9183fu526ohi9pl25k/family1/c5ddc0541ce04cc5a1b0722b78375f45 2023-11-28 03:53:38,685 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__rrgepe6ttujvudbhq99k3mvj7cai9j9fu249ie83lrhvtb9183fu526ohi9pl25k/family1/c5ddc0541ce04cc5a1b0722b78375f45 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9028932752f3f69b0ced4d93db937aaf/family1/6ae1bfa95ab140c6aefb44a69c141607_SeqId_53_ 2023-11-28 03:53:38,686 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HStore(849): Loaded HFile hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__rrgepe6ttujvudbhq99k3mvj7cai9j9fu249ie83lrhvtb9183fu526ohi9pl25k/family1/c5ddc0541ce04cc5a1b0722b78375f45 into 9028932752f3f69b0ced4d93db937aaf/family1 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9028932752f3f69b0ced4d93db937aaf/family1/6ae1bfa95ab140c6aefb44a69c141607_SeqId_53_ - updating store file list. 2023-11-28 03:53:38,692 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HStore(882): Loaded HFile hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9028932752f3f69b0ced4d93db937aaf/family1/6ae1bfa95ab140c6aefb44a69c141607_SeqId_53_ into 9028932752f3f69b0ced4d93db937aaf/family1 2023-11-28 03:53:38,692 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HStore(855): Successfully loaded hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__rrgepe6ttujvudbhq99k3mvj7cai9j9fu249ie83lrhvtb9183fu526ohi9pl25k/family1/c5ddc0541ce04cc5a1b0722b78375f45 into 9028932752f3f69b0ced4d93db937aaf/family1 (new location: hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9028932752f3f69b0ced4d93db937aaf/family1/6ae1bfa95ab140c6aefb44a69c141607_SeqId_53_) 2023-11-28 03:53:38,693 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__rrgepe6ttujvudbhq99k3mvj7cai9j9fu249ie83lrhvtb9183fu526ohi9pl25k/family1/c5ddc0541ce04cc5a1b0722b78375f45 2023-11-28 03:53:38,695 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-11-28 03:53:38,696 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-28 03:53:38,698 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201511201c99ca4147084f1cbf9398911c53ce21 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201511201c99ca4147084f1cbf9398911c53ce21 2023-11-28 03:53:38,700 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201511285b6974886a4643fbba61e9ebbd80aa27 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201511285b6974886a4643fbba61e9ebbd80aa27 2023-11-28 03:53:38,700 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthly 2 expired mob files are deleted 2023-11-28 03:53:38,700 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151101' 2023-11-28 03:53:38,700 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-11-28 03:53:38,701 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-11-28 03:53:38,701 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-11-28 03:53:38,701 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-11-28 03:53:38,701 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-11-28 03:53:38,702 DEBUG [PEWorker-4] locking.LockProcedure(242): UNLOCKED pid=18, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-11-28 03:53:38,704 INFO [PEWorker-4] procedure2.ProcedureExecutor(1409): Finished pid=18, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE in 921 msec 2023-11-28 03:53:38,794 INFO [Listener at localhost/42853] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthly Thread=249 (was 214) Potentially hanging thread: RPCClient-NioEventLoopGroup-4-3 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1016373073_17 at /127.0.0.1:48904 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x398d818a-metaLookup-shared--pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:55641@0x51ad0549-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:549) Potentially hanging thread: RPCClient-NioEventLoopGroup-4-2 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1016373073_17 at /127.0.0.1:48656 [Waiting for operation #13] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-4-4 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: MobCompactor-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1016373073_17 at /127.0.0.1:48842 [Waiting for operation #12] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:55641@0x47660d3d sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$88/1587213957.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: htable-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1016373073_17 at /127.0.0.1:48720 [Waiting for operation #14] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1016373073_17 at /127.0.0.1:48694 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS:0;jenkins-hbase4:41203-shortCompactions-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.PriorityBlockingQueue.take(PriorityBlockingQueue.java:549) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:55641@0x47660d3d-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:549) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1016373073_17 at /127.0.0.1:48892 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:55641@0x51ad0549 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$88/1587213957.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1016373073_17 at /127.0.0.1:48860 [Waiting for operation #13] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1016373073_17 at /127.0.0.1:48712 [Waiting for operation #10] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x53c16fe4-shared-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1016373073_17 at /127.0.0.1:48772 [Waiting for operation #14] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-36928057_17 at /127.0.0.1:48788 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: (jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:55641@0x47660d3d-SendThread(127.0.0.1:55641) sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:332) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1289) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1016373073_17 at /127.0.0.1:48806 [Waiting for operation #10] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-1-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:316) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:373) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1016373073_17 at /127.0.0.1:48836 [Waiting for operation #12] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hfile_cleaner-dir-scan-pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:55641@0x51ad0549-SendThread(127.0.0.1:55641) sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:332) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1289) Potentially hanging thread: hconnection-0x398d818a-shared-pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1016373073_17 at /127.0.0.1:48742 [Waiting for operation #22] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1016373073_17 at /127.0.0.1:48738 [Waiting for operation #16] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x53c16fe4-shared-pool-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1016373073_17 at /127.0.0.1:48722 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Timer for 'HBase' metrics system java.lang.Object.wait(Native Method) java.util.TimerThread.mainLoop(Timer.java:552) java.util.TimerThread.run(Timer.java:505) Potentially hanging thread: RSProcedureDispatcher-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x398d818a-shared-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hfile_cleaner-dir-scan-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=618 (was 505) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=273 (was 304), ProcessCount=172 (was 172), AvailableMemoryMB=7150 (was 7173) 2023-11-28 03:53:38,808 INFO [Listener at localhost/42853] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionWithMonthlyPolicy Thread=249, OpenFileDescriptor=618, MaxFileDescriptor=60000, SystemLoadAverage=273, ProcessCount=172, AvailableMemoryMB=7149 2023-11-28 03:53:38,810 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testMajorCompactionWithMonthlyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} 2023-11-28 03:53:38,812 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] procedure2.ProcedureExecutor(1028): Stored pid=19, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:38,813 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_PRE_OPERATION 2023-11-28 03:53:38,813 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(700): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionWithMonthlyPolicy" procId is: 19 2023-11-28 03:53:38,814 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(1230): Checking to see if procedure is done pid=19 2023-11-28 03:53:38,815 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-11-28 03:53:38,816 DEBUG [HFileArchiver-3] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/.tmp/data/default/testMajorCompactionWithMonthlyPolicy/f40c877e8a9ed20266673cd1013cd07f 2023-11-28 03:53:38,817 DEBUG [HFileArchiver-3] backup.HFileArchiver(153): Directory hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/.tmp/data/default/testMajorCompactionWithMonthlyPolicy/f40c877e8a9ed20266673cd1013cd07f empty. 2023-11-28 03:53:38,818 DEBUG [HFileArchiver-3] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/.tmp/data/default/testMajorCompactionWithMonthlyPolicy/f40c877e8a9ed20266673cd1013cd07f 2023-11-28 03:53:38,818 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(328): Archived testMajorCompactionWithMonthlyPolicy regions 2023-11-28 03:53:38,828 DEBUG [PEWorker-5] util.FSTableDescriptors(570): Wrote into hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/.tmp/data/default/testMajorCompactionWithMonthlyPolicy/.tabledesc/.tableinfo.0000000001 2023-11-28 03:53:38,830 INFO [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(7675): creating {ENCODED => f40c877e8a9ed20266673cd1013cd07f, NAME => 'testMajorCompactionWithMonthlyPolicy,,1701143618810.f40c877e8a9ed20266673cd1013cd07f.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMajorCompactionWithMonthlyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}, regionDir=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/.tmp 2023-11-28 03:53:38,916 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(1230): Checking to see if procedure is done pid=19 2023-11-28 03:53:39,117 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(1230): Checking to see if procedure is done pid=19 2023-11-28 03:53:39,237 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithMonthlyPolicy,,1701143618810.f40c877e8a9ed20266673cd1013cd07f.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-28 03:53:39,237 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1604): Closing f40c877e8a9ed20266673cd1013cd07f, disabling compactions & flushes 2023-11-28 03:53:39,237 INFO [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithMonthlyPolicy,,1701143618810.f40c877e8a9ed20266673cd1013cd07f. 2023-11-28 03:53:39,237 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithMonthlyPolicy,,1701143618810.f40c877e8a9ed20266673cd1013cd07f. 2023-11-28 03:53:39,237 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithMonthlyPolicy,,1701143618810.f40c877e8a9ed20266673cd1013cd07f. after waiting 0 ms 2023-11-28 03:53:39,238 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithMonthlyPolicy,,1701143618810.f40c877e8a9ed20266673cd1013cd07f. 2023-11-28 03:53:39,238 INFO [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1838): Closed testMajorCompactionWithMonthlyPolicy,,1701143618810.f40c877e8a9ed20266673cd1013cd07f. 2023-11-28 03:53:39,238 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1558): Region close journal for f40c877e8a9ed20266673cd1013cd07f: 2023-11-28 03:53:39,241 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_ADD_TO_META 2023-11-28 03:53:39,242 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionWithMonthlyPolicy,,1701143618810.f40c877e8a9ed20266673cd1013cd07f.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1701143619241"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1701143619241"}]},"ts":"1701143619241"} 2023-11-28 03:53:39,243 INFO [PEWorker-5] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-11-28 03:53:39,244 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-11-28 03:53:39,244 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithMonthlyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1701143619244"}]},"ts":"1701143619244"} 2023-11-28 03:53:39,246 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithMonthlyPolicy, state=ENABLING in hbase:meta 2023-11-28 03:53:39,250 INFO [PEWorker-5] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithMonthlyPolicy, region=f40c877e8a9ed20266673cd1013cd07f, ASSIGN}] 2023-11-28 03:53:39,252 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithMonthlyPolicy, region=f40c877e8a9ed20266673cd1013cd07f, ASSIGN 2023-11-28 03:53:39,253 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithMonthlyPolicy, region=f40c877e8a9ed20266673cd1013cd07f, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,41203,1701143600653; forceNewPlan=false, retain=false 2023-11-28 03:53:39,404 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=20 updating hbase:meta row=f40c877e8a9ed20266673cd1013cd07f, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:39,404 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithMonthlyPolicy,,1701143618810.f40c877e8a9ed20266673cd1013cd07f.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1701143619403"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1701143619403"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1701143619403"}]},"ts":"1701143619403"} 2023-11-28 03:53:39,406 INFO [PEWorker-2] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=21, ppid=20, state=RUNNABLE; OpenRegionProcedure f40c877e8a9ed20266673cd1013cd07f, server=jenkins-hbase4.apache.org,41203,1701143600653}] 2023-11-28 03:53:39,418 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(1230): Checking to see if procedure is done pid=19 2023-11-28 03:53:39,558 DEBUG [RSProcedureDispatcher-pool-2] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:39,562 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testMajorCompactionWithMonthlyPolicy,,1701143618810.f40c877e8a9ed20266673cd1013cd07f. 2023-11-28 03:53:39,563 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => f40c877e8a9ed20266673cd1013cd07f, NAME => 'testMajorCompactionWithMonthlyPolicy,,1701143618810.f40c877e8a9ed20266673cd1013cd07f.', STARTKEY => '', ENDKEY => ''} 2023-11-28 03:53:39,563 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithMonthlyPolicy f40c877e8a9ed20266673cd1013cd07f 2023-11-28 03:53:39,563 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithMonthlyPolicy,,1701143618810.f40c877e8a9ed20266673cd1013cd07f.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-28 03:53:39,563 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for f40c877e8a9ed20266673cd1013cd07f 2023-11-28 03:53:39,563 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for f40c877e8a9ed20266673cd1013cd07f 2023-11-28 03:53:39,565 INFO [StoreOpener-f40c877e8a9ed20266673cd1013cd07f-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region f40c877e8a9ed20266673cd1013cd07f 2023-11-28 03:53:39,566 DEBUG [StoreOpener-f40c877e8a9ed20266673cd1013cd07f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithMonthlyPolicy/f40c877e8a9ed20266673cd1013cd07f/family1 2023-11-28 03:53:39,566 DEBUG [StoreOpener-f40c877e8a9ed20266673cd1013cd07f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithMonthlyPolicy/f40c877e8a9ed20266673cd1013cd07f/family1 2023-11-28 03:53:39,567 INFO [StoreOpener-f40c877e8a9ed20266673cd1013cd07f-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region f40c877e8a9ed20266673cd1013cd07f columnFamilyName family1 2023-11-28 03:53:39,568 INFO [StoreOpener-f40c877e8a9ed20266673cd1013cd07f-1] regionserver.HStore(310): Store=f40c877e8a9ed20266673cd1013cd07f/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-28 03:53:39,569 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithMonthlyPolicy/f40c877e8a9ed20266673cd1013cd07f 2023-11-28 03:53:39,569 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithMonthlyPolicy/f40c877e8a9ed20266673cd1013cd07f 2023-11-28 03:53:39,572 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for f40c877e8a9ed20266673cd1013cd07f 2023-11-28 03:53:39,574 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithMonthlyPolicy/f40c877e8a9ed20266673cd1013cd07f/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-11-28 03:53:39,574 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened f40c877e8a9ed20266673cd1013cd07f; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61959318, jitterRate=-0.07673421502113342}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-11-28 03:53:39,575 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for f40c877e8a9ed20266673cd1013cd07f: 2023-11-28 03:53:39,575 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2339): Post open deploy tasks for testMajorCompactionWithMonthlyPolicy,,1701143618810.f40c877e8a9ed20266673cd1013cd07f., pid=21, masterSystemTime=1701143619558 2023-11-28 03:53:39,577 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2366): Finished post open deploy task for testMajorCompactionWithMonthlyPolicy,,1701143618810.f40c877e8a9ed20266673cd1013cd07f. 2023-11-28 03:53:39,578 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testMajorCompactionWithMonthlyPolicy,,1701143618810.f40c877e8a9ed20266673cd1013cd07f. 2023-11-28 03:53:39,578 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=20 updating hbase:meta row=f40c877e8a9ed20266673cd1013cd07f, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:39,578 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithMonthlyPolicy,,1701143618810.f40c877e8a9ed20266673cd1013cd07f.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1701143619578"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1701143619578"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1701143619578"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1701143619578"}]},"ts":"1701143619578"} 2023-11-28 03:53:39,583 INFO [PEWorker-4] procedure2.ProcedureExecutor(1825): Finished subprocedure pid=21, resume processing ppid=20 2023-11-28 03:53:39,583 INFO [PEWorker-4] procedure2.ProcedureExecutor(1409): Finished pid=21, ppid=20, state=SUCCESS; OpenRegionProcedure f40c877e8a9ed20266673cd1013cd07f, server=jenkins-hbase4.apache.org,41203,1701143600653 in 174 msec 2023-11-28 03:53:39,585 INFO [PEWorker-5] procedure2.ProcedureExecutor(1825): Finished subprocedure pid=20, resume processing ppid=19 2023-11-28 03:53:39,585 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=20, ppid=19, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithMonthlyPolicy, region=f40c877e8a9ed20266673cd1013cd07f, ASSIGN in 333 msec 2023-11-28 03:53:39,586 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-11-28 03:53:39,587 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithMonthlyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1701143619586"}]},"ts":"1701143619586"} 2023-11-28 03:53:39,588 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithMonthlyPolicy, state=ENABLED in hbase:meta 2023-11-28 03:53:39,591 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_POST_OPERATION 2023-11-28 03:53:39,593 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=19, state=SUCCESS; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy in 781 msec 2023-11-28 03:53:39,865 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithMonthlyPolicy' 2023-11-28 03:53:39,920 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(1230): Checking to see if procedure is done pid=19 2023-11-28 03:53:39,920 INFO [Listener at localhost/42853] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionWithMonthlyPolicy, procId: 19 completed 2023-11-28 03:53:39,949 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-28 03:53:39,950 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:39,951 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-11-28 03:53:39,951 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:53:39,951 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-11-28 03:53:39,951 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-11-28 03:53:39,952 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:39,952 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:39,953 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:39,953 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:53:39,954 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:53:39,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:53:39,954 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:39,954 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-28 03:53:39,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:39,954 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:39,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-28 03:53:39,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:39,955 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:39,955 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:39,955 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-11-28 03:53:39,955 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:53:39,956 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-11-28 03:53:39,956 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-28 03:53:39,956 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-28 03:53:39,956 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1701143618810.f40c877e8a9ed20266673cd1013cd07f. 2023-11-28 03:53:39,956 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1701143618810.f40c877e8a9ed20266673cd1013cd07f. started... 2023-11-28 03:53:39,957 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing f40c877e8a9ed20266673cd1013cd07f 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-11-28 03:53:39,982 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151130dba525280ff54764b654cb630edd6af6 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151130dba525280ff54764b654cb630edd6af6 2023-11-28 03:53:39,986 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithMonthlyPolicy/f40c877e8a9ed20266673cd1013cd07f/.tmp/family1/05c7e1f03d3b450fa115adc607f67278 2023-11-28 03:53:39,992 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithMonthlyPolicy/f40c877e8a9ed20266673cd1013cd07f/.tmp/family1/05c7e1f03d3b450fa115adc607f67278 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithMonthlyPolicy/f40c877e8a9ed20266673cd1013cd07f/family1/05c7e1f03d3b450fa115adc607f67278 2023-11-28 03:53:39,998 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithMonthlyPolicy/f40c877e8a9ed20266673cd1013cd07f/family1/05c7e1f03d3b450fa115adc607f67278, entries=1000, sequenceid=5, filesize=166.0 K 2023-11-28 03:53:39,999 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for f40c877e8a9ed20266673cd1013cd07f in 42ms, sequenceid=5, compaction requested=false 2023-11-28 03:53:39,999 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for f40c877e8a9ed20266673cd1013cd07f: 2023-11-28 03:53:39,999 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1701143618810.f40c877e8a9ed20266673cd1013cd07f. 2023-11-28 03:53:40,000 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-28 03:53:40,000 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-28 03:53:40,000 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:40,000 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-11-28 03:53:40,000 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-11-28 03:53:40,002 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:40,002 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:40,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:40,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:53:40,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:53:40,002 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:40,002 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-28 03:53:40,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:53:40,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:53:40,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:40,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:40,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:53:40,004 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-11-28 03:53:40,004 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4fc42a0[Count = 0] remaining members to acquire global barrier 2023-11-28 03:53:40,004 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-11-28 03:53:40,004 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:40,006 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:40,006 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:40,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:40,006 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-11-28 03:53:40,006 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-11-28 03:53:40,006 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,41203,1701143600653' in zk 2023-11-28 03:53:40,007 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:40,007 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-28 03:53:40,008 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-11-28 03:53:40,008 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:40,008 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:53:40,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:40,008 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-11-28 03:53:40,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:53:40,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:53:40,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:53:40,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:53:40,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:40,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:40,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:53:40,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:40,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:40,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,41203,1701143600653': 2023-11-28 03:53:40,012 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-11-28 03:53:40,012 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-11-28 03:53:40,012 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-28 03:53:40,012 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-28 03:53:40,012 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:40,012 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-28 03:53:40,014 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:40,014 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:40,014 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:40,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:40,014 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:53:40,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:40,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:53:40,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:53:40,014 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:40,014 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:53:40,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:53:40,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:53:40,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:40,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:40,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:53:40,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:40,016 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:40,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:40,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:53:40,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:40,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:40,024 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:40,025 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:40,025 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:53:40,025 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:40,025 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:53:40,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:53:40,025 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:53:40,025 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:53:40,025 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-11-28 03:53:40,026 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-28 03:53:40,025 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(399): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-11-28 03:53:40,025 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:40,026 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:40,026 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:40,026 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-11-28 03:53:40,026 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-28 03:53:40,026 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:40,026 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:53:40,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:53:40,126 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-28 03:53:40,127 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-28 03:53:40,133 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-28 03:53:40,134 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-11-28 03:53:40,135 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:40,135 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-11-28 03:53:40,135 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:53:40,135 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-11-28 03:53:40,135 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-11-28 03:53:40,136 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:40,136 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:40,137 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:53:40,137 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:40,137 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:53:40,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:53:40,138 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:40,138 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-28 03:53:40,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:40,139 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:40,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-28 03:53:40,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:40,139 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:40,139 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-11-28 03:53:40,139 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:40,139 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-11-28 03:53:40,140 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:53:40,140 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-11-28 03:53:40,140 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-28 03:53:40,140 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-28 03:53:40,140 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1701143618810.f40c877e8a9ed20266673cd1013cd07f. 2023-11-28 03:53:40,140 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1701143618810.f40c877e8a9ed20266673cd1013cd07f. started... 2023-11-28 03:53:40,140 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing f40c877e8a9ed20266673cd1013cd07f 1/1 column families, dataSize=74 B heapSize=408 B 2023-11-28 03:53:40,164 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128917da52aa1ce4736a7d002f4c93b7844 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151128917da52aa1ce4736a7d002f4c93b7844 2023-11-28 03:53:40,571 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithMonthlyPolicy/f40c877e8a9ed20266673cd1013cd07f/.tmp/family1/0642a45db17b47918f51068da30ac824 2023-11-28 03:53:40,578 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithMonthlyPolicy/f40c877e8a9ed20266673cd1013cd07f/.tmp/family1/0642a45db17b47918f51068da30ac824 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithMonthlyPolicy/f40c877e8a9ed20266673cd1013cd07f/family1/0642a45db17b47918f51068da30ac824 2023-11-28 03:53:40,585 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithMonthlyPolicy/f40c877e8a9ed20266673cd1013cd07f/family1/0642a45db17b47918f51068da30ac824, entries=1, sequenceid=9, filesize=5.0 K 2023-11-28 03:53:40,586 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for f40c877e8a9ed20266673cd1013cd07f in 446ms, sequenceid=9, compaction requested=false 2023-11-28 03:53:40,586 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for f40c877e8a9ed20266673cd1013cd07f: 2023-11-28 03:53:40,586 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1701143618810.f40c877e8a9ed20266673cd1013cd07f. 2023-11-28 03:53:40,586 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-28 03:53:40,586 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-28 03:53:40,586 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:40,586 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-11-28 03:53:40,586 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-11-28 03:53:40,588 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:40,588 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:40,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:40,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:53:40,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:53:40,588 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:40,589 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-28 03:53:40,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:53:40,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:53:40,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:40,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:40,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:53:40,590 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-11-28 03:53:40,590 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@74a732a3[Count = 0] remaining members to acquire global barrier 2023-11-28 03:53:40,590 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-11-28 03:53:40,590 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:40,592 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:40,592 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:40,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:40,592 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-11-28 03:53:40,592 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-11-28 03:53:40,592 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,41203,1701143600653' in zk 2023-11-28 03:53:40,592 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:40,592 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-28 03:53:40,595 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:40,595 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-11-28 03:53:40,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:40,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:53:40,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:53:40,595 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:53:40,595 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-11-28 03:53:40,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:53:40,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:53:40,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:40,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:40,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:53:40,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:40,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:40,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,41203,1701143600653': 2023-11-28 03:53:40,598 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-11-28 03:53:40,598 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-11-28 03:53:40,598 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-28 03:53:40,598 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-28 03:53:40,598 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:40,598 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-28 03:53:40,600 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:40,600 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:40,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:40,600 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:40,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:40,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:53:40,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:53:40,600 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:53:40,601 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:53:40,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:53:40,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:53:40,601 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:40,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:40,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:40,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:53:40,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:40,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:40,603 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:40,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:53:40,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:40,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:40,606 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:40,606 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:40,606 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:53:40,606 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:53:40,606 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:53:40,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:53:40,606 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:53:40,606 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:40,606 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-11-28 03:53:40,607 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-28 03:53:40,607 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:40,607 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:40,607 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:40,607 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:40,607 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:53:40,607 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-11-28 03:53:40,607 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-28 03:53:40,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:53:40,707 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-28 03:53:40,709 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-28 03:53:40,714 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-28 03:53:40,715 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-11-28 03:53:40,715 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:40,715 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-11-28 03:53:40,715 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:53:40,716 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-11-28 03:53:40,716 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-11-28 03:53:40,717 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:40,717 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:40,719 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:40,719 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:53:40,719 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:53:40,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:53:40,719 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:40,719 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-28 03:53:40,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:40,720 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:40,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-28 03:53:40,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:40,720 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:40,721 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-11-28 03:53:40,721 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:40,721 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-11-28 03:53:40,721 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:53:40,721 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-11-28 03:53:40,721 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-28 03:53:40,722 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-28 03:53:40,722 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1701143618810.f40c877e8a9ed20266673cd1013cd07f. 2023-11-28 03:53:40,722 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1701143618810.f40c877e8a9ed20266673cd1013cd07f. started... 2023-11-28 03:53:40,722 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing f40c877e8a9ed20266673cd1013cd07f 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-28 03:53:41,138 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512017ebec11d01e34d478a51b1e9f1c0abd0 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201512017ebec11d01e34d478a51b1e9f1c0abd0 2023-11-28 03:53:41,544 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithMonthlyPolicy/f40c877e8a9ed20266673cd1013cd07f/.tmp/family1/37f5cbf2b1794653bef56128bc6f1b4a 2023-11-28 03:53:41,551 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithMonthlyPolicy/f40c877e8a9ed20266673cd1013cd07f/.tmp/family1/37f5cbf2b1794653bef56128bc6f1b4a as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithMonthlyPolicy/f40c877e8a9ed20266673cd1013cd07f/family1/37f5cbf2b1794653bef56128bc6f1b4a 2023-11-28 03:53:41,557 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithMonthlyPolicy/f40c877e8a9ed20266673cd1013cd07f/family1/37f5cbf2b1794653bef56128bc6f1b4a, entries=1, sequenceid=13, filesize=4.9 K 2023-11-28 03:53:41,558 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for f40c877e8a9ed20266673cd1013cd07f in 836ms, sequenceid=13, compaction requested=true 2023-11-28 03:53:41,558 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for f40c877e8a9ed20266673cd1013cd07f: 2023-11-28 03:53:41,558 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1701143618810.f40c877e8a9ed20266673cd1013cd07f. 2023-11-28 03:53:41,558 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-28 03:53:41,558 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-28 03:53:41,558 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:41,558 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-11-28 03:53:41,558 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-11-28 03:53:41,562 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:41,562 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:41,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:41,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:53:41,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:53:41,563 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:41,563 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-28 03:53:41,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:53:41,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:53:41,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:41,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:41,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:53:41,564 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-11-28 03:53:41,564 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@34adbde5[Count = 0] remaining members to acquire global barrier 2023-11-28 03:53:41,564 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-11-28 03:53:41,565 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:41,567 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:41,567 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:41,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:41,568 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-11-28 03:53:41,568 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-11-28 03:53:41,568 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,41203,1701143600653' in zk 2023-11-28 03:53:41,568 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:41,568 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-28 03:53:41,570 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:41,571 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-11-28 03:53:41,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:41,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:53:41,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:53:41,571 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:53:41,571 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-11-28 03:53:41,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:53:41,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:53:41,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:41,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:41,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:53:41,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:41,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:41,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,41203,1701143600653': 2023-11-28 03:53:41,574 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-11-28 03:53:41,574 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-11-28 03:53:41,574 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-28 03:53:41,574 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-28 03:53:41,574 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:41,575 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-28 03:53:41,577 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:41,577 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:41,577 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:41,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:41,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:53:41,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:53:41,577 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:53:41,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:41,578 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:41,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:53:41,578 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:53:41,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:53:41,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:41,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:41,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:53:41,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:41,579 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:41,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:41,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:53:41,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:41,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:41,583 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:41,583 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:41,583 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:41,583 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:41,583 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:41,583 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:41,583 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:41,583 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(399): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-11-28 03:53:41,583 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:53:41,583 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-11-28 03:53:41,584 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:53:41,583 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:53:41,584 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-28 03:53:41,584 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:53:41,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:53:41,585 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-11-28 03:53:41,585 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-28 03:53:41,585 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:53:41,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:53:41,685 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-28 03:53:41,686 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-28 03:53:41,691 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-28 03:53:41,693 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-11-28 03:53:41,693 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:41,693 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-11-28 03:53:41,694 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:53:41,694 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-11-28 03:53:41,694 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-11-28 03:53:41,695 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:41,695 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:41,729 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:41,729 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:53:41,730 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:53:41,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:53:41,730 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:41,730 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-28 03:53:41,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:41,731 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:41,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-28 03:53:41,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:41,731 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:41,731 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-11-28 03:53:41,731 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:41,731 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-11-28 03:53:41,731 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:53:41,732 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-11-28 03:53:41,732 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-28 03:53:41,732 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-28 03:53:41,732 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1701143618810.f40c877e8a9ed20266673cd1013cd07f. 2023-11-28 03:53:41,732 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1701143618810.f40c877e8a9ed20266673cd1013cd07f. started... 2023-11-28 03:53:41,732 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing f40c877e8a9ed20266673cd1013cd07f 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-28 03:53:42,148 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205fb877896fd964bc0b8ae24403f93e98d to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151205fb877896fd964bc0b8ae24403f93e98d 2023-11-28 03:53:42,554 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithMonthlyPolicy/f40c877e8a9ed20266673cd1013cd07f/.tmp/family1/6f78832373d1408e8184d18e4dd4d7c4 2023-11-28 03:53:42,561 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithMonthlyPolicy/f40c877e8a9ed20266673cd1013cd07f/.tmp/family1/6f78832373d1408e8184d18e4dd4d7c4 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithMonthlyPolicy/f40c877e8a9ed20266673cd1013cd07f/family1/6f78832373d1408e8184d18e4dd4d7c4 2023-11-28 03:53:42,566 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithMonthlyPolicy/f40c877e8a9ed20266673cd1013cd07f/family1/6f78832373d1408e8184d18e4dd4d7c4, entries=1, sequenceid=17, filesize=4.9 K 2023-11-28 03:53:42,567 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for f40c877e8a9ed20266673cd1013cd07f in 835ms, sequenceid=17, compaction requested=true 2023-11-28 03:53:42,568 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for f40c877e8a9ed20266673cd1013cd07f: 2023-11-28 03:53:42,568 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1701143618810.f40c877e8a9ed20266673cd1013cd07f. 2023-11-28 03:53:42,568 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-28 03:53:42,568 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-28 03:53:42,568 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:42,568 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-11-28 03:53:42,568 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-11-28 03:53:42,606 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:42,606 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:42,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:42,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:53:42,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:53:42,607 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:42,607 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-28 03:53:42,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:53:42,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:53:42,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:42,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:42,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:53:42,609 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-11-28 03:53:42,609 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1dffceba[Count = 0] remaining members to acquire global barrier 2023-11-28 03:53:42,609 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-11-28 03:53:42,609 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:42,629 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:42,629 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:42,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:42,630 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-11-28 03:53:42,630 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-11-28 03:53:42,630 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,41203,1701143600653' in zk 2023-11-28 03:53:42,630 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:42,630 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-28 03:53:42,719 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:42,719 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-11-28 03:53:42,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:42,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:53:42,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:53:42,720 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:53:42,720 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-11-28 03:53:42,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:53:42,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:53:42,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:42,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:42,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:53:42,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:42,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:42,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,41203,1701143600653': 2023-11-28 03:53:42,723 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-11-28 03:53:42,723 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-11-28 03:53:42,723 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-28 03:53:42,723 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-28 03:53:42,723 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:42,723 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-28 03:53:42,760 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:42,760 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:42,760 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:42,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:42,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:53:42,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:53:42,760 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:53:42,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:42,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:53:42,761 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:53:42,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:53:42,761 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:42,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:42,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:42,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:53:42,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:42,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:42,762 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:42,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:53:42,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:42,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:42,791 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:42,791 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:53:42,791 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:42,791 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:53:42,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:53:42,791 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:42,791 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:53:42,791 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-11-28 03:53:42,791 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-28 03:53:42,791 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:53:42,791 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:42,792 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:42,792 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:53:42,792 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-11-28 03:53:42,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:53:42,792 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:42,792 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-28 03:53:42,792 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:42,893 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-28 03:53:42,893 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-28 03:53:42,898 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-28 03:53:42,900 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-11-28 03:53:42,900 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:42,900 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-11-28 03:53:42,900 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:53:42,901 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-11-28 03:53:42,901 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-11-28 03:53:42,901 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:42,901 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:42,929 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:53:42,929 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:42,929 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:53:42,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:53:42,930 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:42,930 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-28 03:53:42,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:42,930 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:42,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-28 03:53:42,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:42,931 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:42,931 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-11-28 03:53:42,931 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:42,931 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-11-28 03:53:42,931 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:53:42,932 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-11-28 03:53:42,932 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-28 03:53:42,932 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-28 03:53:42,932 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1701143618810.f40c877e8a9ed20266673cd1013cd07f. 2023-11-28 03:53:42,932 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1701143618810.f40c877e8a9ed20266673cd1013cd07f. started... 2023-11-28 03:53:42,932 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing f40c877e8a9ed20266673cd1013cd07f 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-28 03:53:42,941 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-11-28 03:53:42,952 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512285d46a6b499c54887a8a60f22ae1012d3 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201512285d46a6b499c54887a8a60f22ae1012d3 2023-11-28 03:53:42,967 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithMonthlyPolicy/f40c877e8a9ed20266673cd1013cd07f/.tmp/family1/27fd86adcfaf4b5bafb019409447052b 2023-11-28 03:53:42,975 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithMonthlyPolicy/f40c877e8a9ed20266673cd1013cd07f/.tmp/family1/27fd86adcfaf4b5bafb019409447052b as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithMonthlyPolicy/f40c877e8a9ed20266673cd1013cd07f/family1/27fd86adcfaf4b5bafb019409447052b 2023-11-28 03:53:42,980 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithMonthlyPolicy/f40c877e8a9ed20266673cd1013cd07f/family1/27fd86adcfaf4b5bafb019409447052b, entries=1, sequenceid=21, filesize=4.9 K 2023-11-28 03:53:42,981 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for f40c877e8a9ed20266673cd1013cd07f in 49ms, sequenceid=21, compaction requested=true 2023-11-28 03:53:42,982 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for f40c877e8a9ed20266673cd1013cd07f: 2023-11-28 03:53:42,982 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1701143618810.f40c877e8a9ed20266673cd1013cd07f. 2023-11-28 03:53:42,982 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-28 03:53:42,982 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-28 03:53:42,982 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:42,982 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-11-28 03:53:42,982 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-11-28 03:53:43,006 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:43,006 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:43,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:43,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:53:43,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:53:43,007 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:43,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:53:43,007 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-28 03:53:43,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:53:43,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:43,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:43,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:53:43,009 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-11-28 03:53:43,009 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7b7066e7[Count = 0] remaining members to acquire global barrier 2023-11-28 03:53:43,009 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-11-28 03:53:43,009 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:43,060 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:43,061 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:43,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:43,061 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-11-28 03:53:43,061 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:43,061 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-28 03:53:43,061 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-11-28 03:53:43,061 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,41203,1701143600653' in zk 2023-11-28 03:53:43,099 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:43,099 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-11-28 03:53:43,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:43,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:53:43,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:53:43,099 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:53:43,099 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-11-28 03:53:43,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:53:43,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:53:43,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:43,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:43,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:53:43,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:43,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:43,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,41203,1701143600653': 2023-11-28 03:53:43,102 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-11-28 03:53:43,102 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-11-28 03:53:43,102 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-28 03:53:43,102 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-28 03:53:43,102 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:43,103 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-28 03:53:43,160 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:43,160 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:43,160 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:43,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:43,160 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:53:43,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:43,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:53:43,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:53:43,161 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:53:43,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:53:43,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:53:43,161 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:43,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:43,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:43,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:53:43,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:43,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:43,163 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:43,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:53:43,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:43,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:43,191 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:53:43,191 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:43,191 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:53:43,191 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-11-28 03:53:43,191 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-28 03:53:43,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:53:43,191 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:53:43,191 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:43,191 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:53:43,192 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-11-28 03:53:43,192 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:43,192 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-28 03:53:43,192 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:53:43,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:53:43,192 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:43,193 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:43,193 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:43,193 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:43,292 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-28 03:53:43,293 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-28 03:53:43,298 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-28 03:53:43,300 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-11-28 03:53:43,300 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:43,300 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-11-28 03:53:43,300 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:53:43,301 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-11-28 03:53:43,301 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-11-28 03:53:43,301 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:43,301 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:43,303 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:43,303 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:53:43,303 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:53:43,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:53:43,303 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:43,303 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-28 03:53:43,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:43,304 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:43,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-28 03:53:43,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:43,304 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:43,305 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-11-28 03:53:43,305 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:43,305 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-11-28 03:53:43,305 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:53:43,305 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-11-28 03:53:43,306 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-28 03:53:43,306 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-28 03:53:43,306 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1701143618810.f40c877e8a9ed20266673cd1013cd07f. 2023-11-28 03:53:43,306 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1701143618810.f40c877e8a9ed20266673cd1013cd07f. started... 2023-11-28 03:53:43,306 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing f40c877e8a9ed20266673cd1013cd07f 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-28 03:53:43,321 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151231b146cd7350734624a46a66673edded0a to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151231b146cd7350734624a46a66673edded0a 2023-11-28 03:53:43,728 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithMonthlyPolicy/f40c877e8a9ed20266673cd1013cd07f/.tmp/family1/86811704da1c4131a8c0c0276c5b0489 2023-11-28 03:53:43,735 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithMonthlyPolicy/f40c877e8a9ed20266673cd1013cd07f/.tmp/family1/86811704da1c4131a8c0c0276c5b0489 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithMonthlyPolicy/f40c877e8a9ed20266673cd1013cd07f/family1/86811704da1c4131a8c0c0276c5b0489 2023-11-28 03:53:43,740 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithMonthlyPolicy/f40c877e8a9ed20266673cd1013cd07f/family1/86811704da1c4131a8c0c0276c5b0489, entries=1, sequenceid=25, filesize=4.9 K 2023-11-28 03:53:43,742 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for f40c877e8a9ed20266673cd1013cd07f in 436ms, sequenceid=25, compaction requested=true 2023-11-28 03:53:43,742 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for f40c877e8a9ed20266673cd1013cd07f: 2023-11-28 03:53:43,742 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1701143618810.f40c877e8a9ed20266673cd1013cd07f. 2023-11-28 03:53:43,742 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-28 03:53:43,742 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-28 03:53:43,742 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:43,742 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-11-28 03:53:43,742 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-11-28 03:53:43,744 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:43,744 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:43,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:43,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:53:43,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:53:43,744 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:43,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:53:43,744 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-28 03:53:43,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:53:43,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:43,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:43,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:53:43,746 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-11-28 03:53:43,746 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@368f7100[Count = 0] remaining members to acquire global barrier 2023-11-28 03:53:43,746 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-11-28 03:53:43,746 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:43,748 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:43,748 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:43,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:43,748 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-11-28 03:53:43,748 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:43,748 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-28 03:53:43,748 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-11-28 03:53:43,748 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,41203,1701143600653' in zk 2023-11-28 03:53:43,750 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:43,750 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-11-28 03:53:43,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:43,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:53:43,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:53:43,750 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:53:43,750 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-11-28 03:53:43,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:53:43,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:53:43,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:43,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:43,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:53:43,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:43,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:43,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,41203,1701143600653': 2023-11-28 03:53:43,753 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-11-28 03:53:43,753 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-11-28 03:53:43,753 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-28 03:53:43,753 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-28 03:53:43,753 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:43,753 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-28 03:53:43,755 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:43,755 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:43,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:43,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:53:43,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:53:43,755 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:43,755 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:53:43,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:43,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:53:43,756 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:53:43,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:53:43,756 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:43,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:43,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:43,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:53:43,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:43,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:43,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:53:43,761 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:43,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:43,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:43,763 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:43,763 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:43,763 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:43,763 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:43,763 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:53:43,763 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:43,763 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-11-28 03:53:43,763 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-28 03:53:43,763 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:53:43,763 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:43,763 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:53:43,763 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:53:43,764 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:43,764 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-11-28 03:53:43,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:53:43,764 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-28 03:53:43,765 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:53:43,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:53:43,865 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-28 03:53:43,866 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-28 03:53:43,870 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-28 03:53:43,872 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-11-28 03:53:43,872 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:43,872 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-11-28 03:53:43,872 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:53:43,873 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-11-28 03:53:43,873 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-11-28 03:53:43,873 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:43,873 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:43,876 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:43,876 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:53:43,876 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:53:43,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:53:43,876 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:43,876 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-28 03:53:43,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:43,877 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:43,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-28 03:53:43,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:43,877 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:43,877 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-11-28 03:53:43,877 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:43,877 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-11-28 03:53:43,878 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:53:43,878 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-11-28 03:53:43,878 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-28 03:53:43,878 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-28 03:53:43,878 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1701143618810.f40c877e8a9ed20266673cd1013cd07f. 2023-11-28 03:53:43,878 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1701143618810.f40c877e8a9ed20266673cd1013cd07f. started... 2023-11-28 03:53:43,878 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing f40c877e8a9ed20266673cd1013cd07f 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-28 03:53:44,293 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160101153aeeca49d84056a9189cff45c0949a to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20160101153aeeca49d84056a9189cff45c0949a 2023-11-28 03:53:44,699 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithMonthlyPolicy/f40c877e8a9ed20266673cd1013cd07f/.tmp/family1/0b61e1d653d941fca9dac41c5ae432e5 2023-11-28 03:53:44,705 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithMonthlyPolicy/f40c877e8a9ed20266673cd1013cd07f/.tmp/family1/0b61e1d653d941fca9dac41c5ae432e5 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithMonthlyPolicy/f40c877e8a9ed20266673cd1013cd07f/family1/0b61e1d653d941fca9dac41c5ae432e5 2023-11-28 03:53:44,711 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithMonthlyPolicy/f40c877e8a9ed20266673cd1013cd07f/family1/0b61e1d653d941fca9dac41c5ae432e5, entries=1, sequenceid=29, filesize=4.9 K 2023-11-28 03:53:44,712 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for f40c877e8a9ed20266673cd1013cd07f in 834ms, sequenceid=29, compaction requested=true 2023-11-28 03:53:44,712 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for f40c877e8a9ed20266673cd1013cd07f: 2023-11-28 03:53:44,712 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1701143618810.f40c877e8a9ed20266673cd1013cd07f. 2023-11-28 03:53:44,712 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-28 03:53:44,712 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-28 03:53:44,712 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:44,712 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-11-28 03:53:44,712 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-11-28 03:53:44,720 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:44,720 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:44,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:44,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:53:44,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:53:44,721 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:44,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:53:44,721 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-28 03:53:44,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:53:44,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:44,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:44,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:53:44,722 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-11-28 03:53:44,723 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6f94b0a1[Count = 0] remaining members to acquire global barrier 2023-11-28 03:53:44,723 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-11-28 03:53:44,723 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:44,724 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:44,724 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:44,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:44,724 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-11-28 03:53:44,724 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:44,724 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-28 03:53:44,724 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-11-28 03:53:44,724 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,41203,1701143600653' in zk 2023-11-28 03:53:44,726 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:44,726 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-11-28 03:53:44,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:44,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:53:44,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:53:44,726 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:53:44,726 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-11-28 03:53:44,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:53:44,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:53:44,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:44,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:44,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:53:44,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:44,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:44,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,41203,1701143600653': 2023-11-28 03:53:44,729 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-11-28 03:53:44,729 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-11-28 03:53:44,729 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-28 03:53:44,729 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-28 03:53:44,729 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:44,729 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-28 03:53:44,731 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:44,731 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:44,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:44,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:53:44,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:53:44,731 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:53:44,731 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:44,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:44,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:53:44,731 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:53:44,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:53:44,731 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:44,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:44,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:44,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:53:44,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:44,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:44,733 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:44,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:53:44,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:44,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:44,737 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:44,737 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:44,737 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:53:44,737 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:44,737 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:53:44,737 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:53:44,737 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:53:44,737 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-11-28 03:53:44,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:53:44,737 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:44,738 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-28 03:53:44,738 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:44,738 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:44,738 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:44,738 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:53:44,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:53:44,738 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-11-28 03:53:44,738 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-28 03:53:44,838 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-28 03:53:44,839 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-28 03:53:44,845 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-28 03:53:44,847 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-11-28 03:53:44,847 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:44,847 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-11-28 03:53:44,847 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:53:44,848 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-11-28 03:53:44,848 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-11-28 03:53:44,848 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:44,849 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:44,850 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:44,850 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:53:44,850 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:53:44,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:53:44,850 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:44,850 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-28 03:53:44,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:44,851 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:44,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-28 03:53:44,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:44,851 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:44,852 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-11-28 03:53:44,852 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:44,853 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-11-28 03:53:44,853 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:53:44,853 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-11-28 03:53:44,853 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-28 03:53:44,853 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-28 03:53:44,853 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1701143618810.f40c877e8a9ed20266673cd1013cd07f. 2023-11-28 03:53:44,853 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1701143618810.f40c877e8a9ed20266673cd1013cd07f. started... 2023-11-28 03:53:44,853 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing f40c877e8a9ed20266673cd1013cd07f 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-28 03:53:45,269 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103b7a3737db5f54d2c835196d875556b7f to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20160103b7a3737db5f54d2c835196d875556b7f 2023-11-28 03:53:45,275 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithMonthlyPolicy/f40c877e8a9ed20266673cd1013cd07f/.tmp/family1/09765f32b2ab48f99943c02962e694bc 2023-11-28 03:53:45,280 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithMonthlyPolicy/f40c877e8a9ed20266673cd1013cd07f/.tmp/family1/09765f32b2ab48f99943c02962e694bc as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithMonthlyPolicy/f40c877e8a9ed20266673cd1013cd07f/family1/09765f32b2ab48f99943c02962e694bc 2023-11-28 03:53:45,285 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithMonthlyPolicy/f40c877e8a9ed20266673cd1013cd07f/family1/09765f32b2ab48f99943c02962e694bc, entries=1, sequenceid=33, filesize=4.9 K 2023-11-28 03:53:45,286 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for f40c877e8a9ed20266673cd1013cd07f in 433ms, sequenceid=33, compaction requested=true 2023-11-28 03:53:45,286 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for f40c877e8a9ed20266673cd1013cd07f: 2023-11-28 03:53:45,286 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1701143618810.f40c877e8a9ed20266673cd1013cd07f. 2023-11-28 03:53:45,286 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-28 03:53:45,286 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-28 03:53:45,286 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:45,286 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-11-28 03:53:45,286 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-11-28 03:53:45,288 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:45,288 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:45,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:45,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:53:45,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:53:45,288 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:45,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:53:45,288 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-28 03:53:45,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:53:45,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:45,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:45,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:53:45,290 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-11-28 03:53:45,290 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@cfd39f7[Count = 0] remaining members to acquire global barrier 2023-11-28 03:53:45,290 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-11-28 03:53:45,290 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:45,291 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:45,291 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:45,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:45,291 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-11-28 03:53:45,291 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-11-28 03:53:45,292 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,41203,1701143600653' in zk 2023-11-28 03:53:45,292 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:45,292 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-28 03:53:45,294 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:45,294 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-11-28 03:53:45,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:45,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:53:45,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:53:45,294 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:53:45,294 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-11-28 03:53:45,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:53:45,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:53:45,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:45,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:45,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:53:45,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:45,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:45,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,41203,1701143600653': 2023-11-28 03:53:45,298 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-11-28 03:53:45,298 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-11-28 03:53:45,298 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-28 03:53:45,298 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-28 03:53:45,298 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:45,298 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-28 03:53:45,299 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:45,299 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:45,299 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:53:45,299 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:45,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:45,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:45,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:53:45,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:53:45,300 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:53:45,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:53:45,300 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:45,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:53:45,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:45,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:45,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:53:45,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:45,302 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:45,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:45,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:53:45,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:45,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:45,305 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:45,305 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:53:45,305 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:45,305 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:53:45,305 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:45,305 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:45,305 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:45,305 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:45,305 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:53:45,305 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(399): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-11-28 03:53:45,305 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:53:45,305 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-11-28 03:53:45,306 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-28 03:53:45,305 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:45,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:53:45,306 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-11-28 03:53:45,306 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:53:45,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:53:45,306 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-28 03:53:45,407 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-28 03:53:45,408 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-28 03:53:45,413 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-28 03:53:45,415 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-11-28 03:53:45,415 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:45,415 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-11-28 03:53:45,415 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:53:45,415 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-11-28 03:53:45,415 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-11-28 03:53:45,416 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:45,416 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:45,417 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:45,417 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:53:45,418 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:53:45,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:53:45,418 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:45,418 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-28 03:53:45,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:45,418 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:45,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-28 03:53:45,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:45,420 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:45,420 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-11-28 03:53:45,420 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:45,420 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-11-28 03:53:45,420 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:53:45,421 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-11-28 03:53:45,421 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-28 03:53:45,421 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-28 03:53:45,421 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1701143618810.f40c877e8a9ed20266673cd1013cd07f. 2023-11-28 03:53:45,421 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1701143618810.f40c877e8a9ed20266673cd1013cd07f. started... 2023-11-28 03:53:45,421 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing f40c877e8a9ed20266673cd1013cd07f 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-28 03:53:45,439 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20150907786e1780b0e84eac89664ffd9d429c23 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20150907786e1780b0e84eac89664ffd9d429c23 2023-11-28 03:53:45,846 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithMonthlyPolicy/f40c877e8a9ed20266673cd1013cd07f/.tmp/family1/d61efffc63eb45a894012197f03f05f0 2023-11-28 03:53:45,853 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithMonthlyPolicy/f40c877e8a9ed20266673cd1013cd07f/.tmp/family1/d61efffc63eb45a894012197f03f05f0 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithMonthlyPolicy/f40c877e8a9ed20266673cd1013cd07f/family1/d61efffc63eb45a894012197f03f05f0 2023-11-28 03:53:45,858 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithMonthlyPolicy/f40c877e8a9ed20266673cd1013cd07f/family1/d61efffc63eb45a894012197f03f05f0, entries=1, sequenceid=37, filesize=4.9 K 2023-11-28 03:53:45,859 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for f40c877e8a9ed20266673cd1013cd07f in 438ms, sequenceid=37, compaction requested=true 2023-11-28 03:53:45,859 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for f40c877e8a9ed20266673cd1013cd07f: 2023-11-28 03:53:45,859 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1701143618810.f40c877e8a9ed20266673cd1013cd07f. 2023-11-28 03:53:45,859 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-28 03:53:45,859 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-28 03:53:45,859 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:45,859 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-11-28 03:53:45,859 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-11-28 03:53:45,863 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:45,863 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:45,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:45,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:53:45,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:53:45,863 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:45,863 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-28 03:53:45,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:53:45,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:53:45,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:45,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:45,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:53:45,864 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-11-28 03:53:45,865 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@744221d1[Count = 0] remaining members to acquire global barrier 2023-11-28 03:53:45,865 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-11-28 03:53:45,865 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:45,866 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:45,866 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:45,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:45,866 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-11-28 03:53:45,866 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-11-28 03:53:45,866 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,41203,1701143600653' in zk 2023-11-28 03:53:45,866 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:45,866 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-28 03:53:45,868 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:45,868 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-11-28 03:53:45,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:45,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:53:45,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:53:45,868 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:53:45,868 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-11-28 03:53:45,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:53:45,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:53:45,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:45,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:45,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:53:45,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:45,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:45,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,41203,1701143600653': 2023-11-28 03:53:45,871 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-11-28 03:53:45,871 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-11-28 03:53:45,871 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-28 03:53:45,871 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-28 03:53:45,871 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:45,871 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-28 03:53:45,873 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:45,873 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:53:45,873 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:45,873 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:45,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:45,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:45,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:53:45,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:53:45,873 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:53:45,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:53:45,873 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:45,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:53:45,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:45,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:45,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:53:45,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:45,875 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:45,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:45,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:53:45,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:45,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:45,880 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:45,880 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:45,880 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:45,880 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:45,880 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:53:45,880 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:45,881 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:53:45,881 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(399): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-11-28 03:53:45,880 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:53:45,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:53:45,881 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-11-28 03:53:45,881 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-28 03:53:45,881 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:45,881 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:53:45,881 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:45,882 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:53:45,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:53:45,882 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-11-28 03:53:45,882 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-28 03:53:45,983 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-28 03:53:45,984 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-28 03:53:45,989 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-28 03:53:45,990 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-11-28 03:53:45,990 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:45,990 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-11-28 03:53:45,990 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:53:45,991 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-11-28 03:53:45,991 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-11-28 03:53:45,991 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:45,991 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:45,993 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:45,993 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:53:45,993 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:53:45,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:53:45,993 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:45,993 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-28 03:53:45,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:45,994 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:45,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-28 03:53:45,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:45,994 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:45,995 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-11-28 03:53:45,995 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:45,995 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-11-28 03:53:45,995 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:53:45,996 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-11-28 03:53:45,996 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-28 03:53:45,996 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-28 03:53:45,996 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1701143618810.f40c877e8a9ed20266673cd1013cd07f. 2023-11-28 03:53:45,996 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1701143618810.f40c877e8a9ed20266673cd1013cd07f. started... 2023-11-28 03:53:45,996 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing f40c877e8a9ed20266673cd1013cd07f 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-28 03:53:46,010 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151120561c856e2f46400bb66e4929aa27d57d to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151120561c856e2f46400bb66e4929aa27d57d 2023-11-28 03:53:46,416 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithMonthlyPolicy/f40c877e8a9ed20266673cd1013cd07f/.tmp/family1/1c16386b44e8460ebc1722d036f73ddb 2023-11-28 03:53:46,422 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithMonthlyPolicy/f40c877e8a9ed20266673cd1013cd07f/.tmp/family1/1c16386b44e8460ebc1722d036f73ddb as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithMonthlyPolicy/f40c877e8a9ed20266673cd1013cd07f/family1/1c16386b44e8460ebc1722d036f73ddb 2023-11-28 03:53:46,428 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithMonthlyPolicy/f40c877e8a9ed20266673cd1013cd07f/family1/1c16386b44e8460ebc1722d036f73ddb, entries=1, sequenceid=41, filesize=4.9 K 2023-11-28 03:53:46,429 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for f40c877e8a9ed20266673cd1013cd07f in 433ms, sequenceid=41, compaction requested=true 2023-11-28 03:53:46,429 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for f40c877e8a9ed20266673cd1013cd07f: 2023-11-28 03:53:46,429 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1701143618810.f40c877e8a9ed20266673cd1013cd07f. 2023-11-28 03:53:46,429 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-28 03:53:46,429 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-28 03:53:46,429 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:46,429 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-11-28 03:53:46,429 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-11-28 03:53:46,431 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:46,431 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:46,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:46,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:53:46,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:53:46,431 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:46,431 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-28 03:53:46,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:53:46,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:53:46,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:46,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:46,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:53:46,433 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-11-28 03:53:46,433 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@39884928[Count = 0] remaining members to acquire global barrier 2023-11-28 03:53:46,433 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-11-28 03:53:46,433 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:46,434 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:46,434 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:46,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:46,434 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-11-28 03:53:46,434 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-11-28 03:53:46,434 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:46,435 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-28 03:53:46,435 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,41203,1701143600653' in zk 2023-11-28 03:53:46,437 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:46,437 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-11-28 03:53:46,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:46,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:53:46,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:53:46,437 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:53:46,437 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-11-28 03:53:46,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:53:46,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:53:46,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:46,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:46,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:53:46,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:46,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:46,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,41203,1701143600653': 2023-11-28 03:53:46,440 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-11-28 03:53:46,440 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-11-28 03:53:46,440 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-28 03:53:46,440 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-28 03:53:46,440 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:46,440 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-28 03:53:46,442 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:46,442 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:53:46,442 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:46,442 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:46,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:46,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:46,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:53:46,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:53:46,442 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:53:46,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:53:46,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:53:46,443 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:46,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:46,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:46,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:53:46,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:46,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:46,444 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:46,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:53:46,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:46,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:46,447 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:46,447 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:46,447 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:46,447 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:46,447 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:53:46,447 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:46,447 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:53:46,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:53:46,447 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:53:46,447 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-11-28 03:53:46,447 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-28 03:53:46,447 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:53:46,447 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:46,447 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:53:46,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:53:46,447 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:46,447 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-11-28 03:53:46,448 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-28 03:53:46,548 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-28 03:53:46,549 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-28 03:53:46,550 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=38593] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}] of the table testMajorCompactionWithMonthlyPolicy 2023-11-28 03:53:46,552 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] procedure2.ProcedureExecutor(1028): Stored pid=22, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-11-28 03:53:46,553 DEBUG [PEWorker-2] locking.LockProcedure(309): LOCKED pid=22, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-11-28 03:53:46,554 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithMonthlyPolicy', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-11-28 03:53:46,554 INFO [PEWorker-2] procedure2.TimeoutExecutorThread(81): ADDED pid=22, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1701144226554 2023-11-28 03:53:46,555 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-11-28 03:53:46,556 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMajorCompactionWithMonthlyPolicy' and column 'family1' 2023-11-28 03:53:46,556 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-11-28 03:53:46,557 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] zookeeper.ReadOnlyZKClient(139): Connect 0x56d83728 to 127.0.0.1:55641 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-11-28 03:53:46,562 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] ipc.AbstractRpcClient(189): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@443b7b0c, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-11-28 03:53:46,562 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-11-28 03:53:47,399 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103f64f0e5983574dcab163b061e8b199ad to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20160103f64f0e5983574dcab163b061e8b199ad 2023-11-28 03:53:47,400 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20160101 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-11-28 03:53:47,415 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(122): Using SIMPLE authentication for service=ClientService, sasl=false 2023-11-28 03:53:47,416 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:58768, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-11-28 03:53:47,423 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(122): Using SIMPLE authentication for service=MasterService, sasl=false 2023-11-28 03:53:47,424 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:55164, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-11-28 03:53:47,435 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/b76cebe99eb347748d316fafc3dc24f2 first=Optional[r5] last=Optional[r6] 2023-11-28 03:53:47,435 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithMonthlyPolicy,,1701143618810.f40c877e8a9ed20266673cd1013cd07f., hostname=jenkins-hbase4.apache.org,41203,1701143600653, seqNum=2 for row with hfile group [{family1,hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/b76cebe99eb347748d316fafc3dc24f2}] 2023-11-28 03:53:47,440 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HStore(763): Validating hfile at hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/b76cebe99eb347748d316fafc3dc24f2 for inclusion in f40c877e8a9ed20266673cd1013cd07f/family1 2023-11-28 03:53:47,443 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HStore(775): HFile bounds: first=r5 last=r6 2023-11-28 03:53:47,443 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HStore(777): Region bounds: first= last= 2023-11-28 03:53:47,443 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HRegion(2446): Flush status journal for f40c877e8a9ed20266673cd1013cd07f: 2023-11-28 03:53:47,444 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/b76cebe99eb347748d316fafc3dc24f2 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testMajorCompactionWithMonthlyPolicy__psamg8v6t2ivokfeiiicsi3cvbjdjikunbvfgu47ug2vivguseq70eokgja9mjj2/family1/b76cebe99eb347748d316fafc3dc24f2 2023-11-28 03:53:47,447 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testMajorCompactionWithMonthlyPolicy__psamg8v6t2ivokfeiiicsi3cvbjdjikunbvfgu47ug2vivguseq70eokgja9mjj2/family1/b76cebe99eb347748d316fafc3dc24f2 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithMonthlyPolicy/f40c877e8a9ed20266673cd1013cd07f/family1/18c80b53fbbe4f2e9e2c76311af62902_SeqId_44_ 2023-11-28 03:53:47,447 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HStore(849): Loaded HFile hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testMajorCompactionWithMonthlyPolicy__psamg8v6t2ivokfeiiicsi3cvbjdjikunbvfgu47ug2vivguseq70eokgja9mjj2/family1/b76cebe99eb347748d316fafc3dc24f2 into f40c877e8a9ed20266673cd1013cd07f/family1 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithMonthlyPolicy/f40c877e8a9ed20266673cd1013cd07f/family1/18c80b53fbbe4f2e9e2c76311af62902_SeqId_44_ - updating store file list. 2023-11-28 03:53:47,452 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HStore(882): Loaded HFile hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithMonthlyPolicy/f40c877e8a9ed20266673cd1013cd07f/family1/18c80b53fbbe4f2e9e2c76311af62902_SeqId_44_ into f40c877e8a9ed20266673cd1013cd07f/family1 2023-11-28 03:53:47,452 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HStore(855): Successfully loaded hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testMajorCompactionWithMonthlyPolicy__psamg8v6t2ivokfeiiicsi3cvbjdjikunbvfgu47ug2vivguseq70eokgja9mjj2/family1/b76cebe99eb347748d316fafc3dc24f2 into f40c877e8a9ed20266673cd1013cd07f/family1 (new location: hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithMonthlyPolicy/f40c877e8a9ed20266673cd1013cd07f/family1/18c80b53fbbe4f2e9e2c76311af62902_SeqId_44_) 2023-11-28 03:53:47,453 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testMajorCompactionWithMonthlyPolicy__psamg8v6t2ivokfeiiicsi3cvbjdjikunbvfgu47ug2vivguseq70eokgja9mjj2/family1/b76cebe99eb347748d316fafc3dc24f2 2023-11-28 03:53:47,453 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-11-28 03:53:47,457 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 50903 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-11-28 03:53:47,457 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HStore(1912): f40c877e8a9ed20266673cd1013cd07f/family1 is initiating minor compaction 2023-11-28 03:53:47,457 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@23f5e790; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-28 03:53:47,457 INFO [RS:0;jenkins-hbase4:41203-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of f40c877e8a9ed20266673cd1013cd07f/family1 in testMajorCompactionWithMonthlyPolicy,,1701143618810.f40c877e8a9ed20266673cd1013cd07f. 2023-11-28 03:53:47,457 INFO [RS:0;jenkins-hbase4:41203-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithMonthlyPolicy/f40c877e8a9ed20266673cd1013cd07f/family1/0642a45db17b47918f51068da30ac824, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithMonthlyPolicy/f40c877e8a9ed20266673cd1013cd07f/family1/37f5cbf2b1794653bef56128bc6f1b4a, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithMonthlyPolicy/f40c877e8a9ed20266673cd1013cd07f/family1/6f78832373d1408e8184d18e4dd4d7c4, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithMonthlyPolicy/f40c877e8a9ed20266673cd1013cd07f/family1/27fd86adcfaf4b5bafb019409447052b, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithMonthlyPolicy/f40c877e8a9ed20266673cd1013cd07f/family1/86811704da1c4131a8c0c0276c5b0489, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithMonthlyPolicy/f40c877e8a9ed20266673cd1013cd07f/family1/0b61e1d653d941fca9dac41c5ae432e5, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithMonthlyPolicy/f40c877e8a9ed20266673cd1013cd07f/family1/09765f32b2ab48f99943c02962e694bc, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithMonthlyPolicy/f40c877e8a9ed20266673cd1013cd07f/family1/d61efffc63eb45a894012197f03f05f0, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithMonthlyPolicy/f40c877e8a9ed20266673cd1013cd07f/family1/1c16386b44e8460ebc1722d036f73ddb, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithMonthlyPolicy/f40c877e8a9ed20266673cd1013cd07f/family1/18c80b53fbbe4f2e9e2c76311af62902_SeqId_44_] into tmpdir=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithMonthlyPolicy/f40c877e8a9ed20266673cd1013cd07f/.tmp, totalSize=49.7 K 2023-11-28 03:53:47,457 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] compactions.Compactor(207): Compacting 0642a45db17b47918f51068da30ac824, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-11-28 03:53:47,457 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] compactions.Compactor(207): Compacting 37f5cbf2b1794653bef56128bc6f1b4a, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=13 2023-11-28 03:53:47,457 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] compactions.Compactor(207): Compacting 6f78832373d1408e8184d18e4dd4d7c4, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=17 2023-11-28 03:53:47,457 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] compactions.Compactor(207): Compacting 27fd86adcfaf4b5bafb019409447052b, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=21 2023-11-28 03:53:47,457 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] compactions.Compactor(207): Compacting 86811704da1c4131a8c0c0276c5b0489, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=25 2023-11-28 03:53:47,457 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] compactions.Compactor(207): Compacting 0b61e1d653d941fca9dac41c5ae432e5, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=29 2023-11-28 03:53:47,458 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] compactions.Compactor(207): Compacting 09765f32b2ab48f99943c02962e694bc, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=33 2023-11-28 03:53:47,458 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] compactions.Compactor(207): Compacting d61efffc63eb45a894012197f03f05f0, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=37 2023-11-28 03:53:47,458 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] compactions.Compactor(207): Compacting 1c16386b44e8460ebc1722d036f73ddb, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=41 2023-11-28 03:53:47,458 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] compactions.Compactor(207): Compacting 18c80b53fbbe4f2e9e2c76311af62902_SeqId_44_, keycount=2, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=45 2023-11-28 03:53:47,459 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20160101 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-11-28 03:53:47,460 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-28 03:53:47,462 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20160101153aeeca49d84056a9189cff45c0949a to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20160101153aeeca49d84056a9189cff45c0949a 2023-11-28 03:53:47,463 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20160103b7a3737db5f54d2c835196d875556b7f to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20160103b7a3737db5f54d2c835196d875556b7f 2023-11-28 03:53:47,463 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithMonthlyPolicy 2 expired mob files are deleted 2023-11-28 03:53:47,463 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101' 2023-11-28 03:53:47,464 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-11-28 03:53:47,464 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-11-28 03:53:47,496 INFO [RS:0;jenkins-hbase4:41203-shortCompactions-0] throttle.PressureAwareThroughputController(145): f40c877e8a9ed20266673cd1013cd07f#family1#compaction#22 average throughput is 0.34 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-11-28 03:53:48,315 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithMonthlyPolicy/f40c877e8a9ed20266673cd1013cd07f/.tmp/family1/9d94a50280d647a9a92ce19b916a1269 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithMonthlyPolicy/f40c877e8a9ed20266673cd1013cd07f/family1/9d94a50280d647a9a92ce19b916a1269 2023-11-28 03:53:48,322 INFO [RS:0;jenkins-hbase4:41203-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in f40c877e8a9ed20266673cd1013cd07f/family1 of f40c877e8a9ed20266673cd1013cd07f into 9d94a50280d647a9a92ce19b916a1269(size=6.6 K), total size for store is 172.6 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-11-28 03:53:48,323 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for f40c877e8a9ed20266673cd1013cd07f: 2023-11-28 03:53:48,323 INFO [RS:0;jenkins-hbase4:41203-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithMonthlyPolicy,,1701143618810.f40c877e8a9ed20266673cd1013cd07f., storeName=family1, fileCount=10, fileSize=49.7 K (5.0 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 5.2 K), priority=2, time=1701143627457; duration=0sec 2023-11-28 03:53:48,323 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-28 03:53:48,323 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151231a29b6d4dcc3d40e9b3c406ef560e4524 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151231a29b6d4dcc3d40e9b3c406ef560e4524 2023-11-28 03:53:48,324 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151201 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-11-28 03:53:48,351 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/fecaca015548462ab5fe1b22006fd63b first=Optional[r1] last=Optional[r4] 2023-11-28 03:53:48,352 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithMonthlyPolicy,,1701143618810.f40c877e8a9ed20266673cd1013cd07f., hostname=jenkins-hbase4.apache.org,41203,1701143600653, seqNum=2 for row with hfile group [{family1,hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/fecaca015548462ab5fe1b22006fd63b}] 2023-11-28 03:53:48,355 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HStore(763): Validating hfile at hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/fecaca015548462ab5fe1b22006fd63b for inclusion in f40c877e8a9ed20266673cd1013cd07f/family1 2023-11-28 03:53:48,359 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HStore(775): HFile bounds: first=r1 last=r4 2023-11-28 03:53:48,359 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HStore(777): Region bounds: first= last= 2023-11-28 03:53:48,359 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HRegion(2446): Flush status journal for f40c877e8a9ed20266673cd1013cd07f: 2023-11-28 03:53:48,360 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/fecaca015548462ab5fe1b22006fd63b to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testMajorCompactionWithMonthlyPolicy__k2lvduopqn3aup1toljlciolhm02ci4rtek5sf617rrt2po9mugdv0av3j95fjbc/family1/fecaca015548462ab5fe1b22006fd63b 2023-11-28 03:53:48,363 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testMajorCompactionWithMonthlyPolicy__k2lvduopqn3aup1toljlciolhm02ci4rtek5sf617rrt2po9mugdv0av3j95fjbc/family1/fecaca015548462ab5fe1b22006fd63b as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithMonthlyPolicy/f40c877e8a9ed20266673cd1013cd07f/family1/f546da9f7fd640b6b02aa6c8f72bd361_SeqId_47_ 2023-11-28 03:53:48,364 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HStore(849): Loaded HFile hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testMajorCompactionWithMonthlyPolicy__k2lvduopqn3aup1toljlciolhm02ci4rtek5sf617rrt2po9mugdv0av3j95fjbc/family1/fecaca015548462ab5fe1b22006fd63b into f40c877e8a9ed20266673cd1013cd07f/family1 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithMonthlyPolicy/f40c877e8a9ed20266673cd1013cd07f/family1/f546da9f7fd640b6b02aa6c8f72bd361_SeqId_47_ - updating store file list. 2023-11-28 03:53:48,369 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HStore(882): Loaded HFile hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithMonthlyPolicy/f40c877e8a9ed20266673cd1013cd07f/family1/f546da9f7fd640b6b02aa6c8f72bd361_SeqId_47_ into f40c877e8a9ed20266673cd1013cd07f/family1 2023-11-28 03:53:48,369 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HStore(855): Successfully loaded hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testMajorCompactionWithMonthlyPolicy__k2lvduopqn3aup1toljlciolhm02ci4rtek5sf617rrt2po9mugdv0av3j95fjbc/family1/fecaca015548462ab5fe1b22006fd63b into f40c877e8a9ed20266673cd1013cd07f/family1 (new location: hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithMonthlyPolicy/f40c877e8a9ed20266673cd1013cd07f/family1/f546da9f7fd640b6b02aa6c8f72bd361_SeqId_47_) 2023-11-28 03:53:48,369 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testMajorCompactionWithMonthlyPolicy__k2lvduopqn3aup1toljlciolhm02ci4rtek5sf617rrt2po9mugdv0av3j95fjbc/family1/fecaca015548462ab5fe1b22006fd63b 2023-11-28 03:53:48,370 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-11-28 03:53:48,371 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 182349 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-11-28 03:53:48,371 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HStore(1912): f40c877e8a9ed20266673cd1013cd07f/family1 is initiating minor compaction (all files) 2023-11-28 03:53:48,371 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@3e0b04b8; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-28 03:53:48,371 INFO [RS:0;jenkins-hbase4:41203-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of f40c877e8a9ed20266673cd1013cd07f/family1 in testMajorCompactionWithMonthlyPolicy,,1701143618810.f40c877e8a9ed20266673cd1013cd07f. 2023-11-28 03:53:48,371 INFO [RS:0;jenkins-hbase4:41203-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithMonthlyPolicy/f40c877e8a9ed20266673cd1013cd07f/family1/05c7e1f03d3b450fa115adc607f67278, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithMonthlyPolicy/f40c877e8a9ed20266673cd1013cd07f/family1/9d94a50280d647a9a92ce19b916a1269, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithMonthlyPolicy/f40c877e8a9ed20266673cd1013cd07f/family1/f546da9f7fd640b6b02aa6c8f72bd361_SeqId_47_] into tmpdir=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithMonthlyPolicy/f40c877e8a9ed20266673cd1013cd07f/.tmp, totalSize=178.1 K 2023-11-28 03:53:48,372 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] compactions.Compactor(207): Compacting 05c7e1f03d3b450fa115adc607f67278, keycount=1000, bloomtype=ROW, size=166.0 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878824053 2023-11-28 03:53:48,372 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] compactions.Compactor(207): Compacting 9d94a50280d647a9a92ce19b916a1269, keycount=9, bloomtype=ROW, size=6.6 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621224053 2023-11-28 03:53:48,373 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] compactions.Compactor(207): Compacting f546da9f7fd640b6b02aa6c8f72bd361_SeqId_47_, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448965224053 2023-11-28 03:53:48,373 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151201 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-11-28 03:53:48,374 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-28 03:53:48,376 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201512017ebec11d01e34d478a51b1e9f1c0abd0 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201512017ebec11d01e34d478a51b1e9f1c0abd0 2023-11-28 03:53:48,378 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151205fb877896fd964bc0b8ae24403f93e98d to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151205fb877896fd964bc0b8ae24403f93e98d 2023-11-28 03:53:48,389 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201512285d46a6b499c54887a8a60f22ae1012d3 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201512285d46a6b499c54887a8a60f22ae1012d3 2023-11-28 03:53:48,390 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithMonthlyPolicy/f40c877e8a9ed20266673cd1013cd07f/family1/0642a45db17b47918f51068da30ac824, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithMonthlyPolicy/f40c877e8a9ed20266673cd1013cd07f/family1/37f5cbf2b1794653bef56128bc6f1b4a, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithMonthlyPolicy/f40c877e8a9ed20266673cd1013cd07f/family1/6f78832373d1408e8184d18e4dd4d7c4, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithMonthlyPolicy/f40c877e8a9ed20266673cd1013cd07f/family1/27fd86adcfaf4b5bafb019409447052b, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithMonthlyPolicy/f40c877e8a9ed20266673cd1013cd07f/family1/86811704da1c4131a8c0c0276c5b0489, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithMonthlyPolicy/f40c877e8a9ed20266673cd1013cd07f/family1/0b61e1d653d941fca9dac41c5ae432e5, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithMonthlyPolicy/f40c877e8a9ed20266673cd1013cd07f/family1/09765f32b2ab48f99943c02962e694bc, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithMonthlyPolicy/f40c877e8a9ed20266673cd1013cd07f/family1/d61efffc63eb45a894012197f03f05f0, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithMonthlyPolicy/f40c877e8a9ed20266673cd1013cd07f/family1/1c16386b44e8460ebc1722d036f73ddb, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithMonthlyPolicy/f40c877e8a9ed20266673cd1013cd07f/family1/18c80b53fbbe4f2e9e2c76311af62902_SeqId_44_] to archive 2023-11-28 03:53:48,391 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-28 03:53:48,392 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151231b146cd7350734624a46a66673edded0a to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151231b146cd7350734624a46a66673edded0a 2023-11-28 03:53:48,392 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithMonthlyPolicy 4 expired mob files are deleted 2023-11-28 03:53:48,392 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201' 2023-11-28 03:53:48,394 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithMonthlyPolicy/f40c877e8a9ed20266673cd1013cd07f/family1/0642a45db17b47918f51068da30ac824 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionWithMonthlyPolicy/f40c877e8a9ed20266673cd1013cd07f/family1/0642a45db17b47918f51068da30ac824 2023-11-28 03:53:48,396 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithMonthlyPolicy/f40c877e8a9ed20266673cd1013cd07f/family1/37f5cbf2b1794653bef56128bc6f1b4a to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionWithMonthlyPolicy/f40c877e8a9ed20266673cd1013cd07f/family1/37f5cbf2b1794653bef56128bc6f1b4a 2023-11-28 03:53:48,396 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-11-28 03:53:48,396 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-11-28 03:53:48,397 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithMonthlyPolicy/f40c877e8a9ed20266673cd1013cd07f/family1/6f78832373d1408e8184d18e4dd4d7c4 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionWithMonthlyPolicy/f40c877e8a9ed20266673cd1013cd07f/family1/6f78832373d1408e8184d18e4dd4d7c4 2023-11-28 03:53:48,399 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithMonthlyPolicy/f40c877e8a9ed20266673cd1013cd07f/family1/27fd86adcfaf4b5bafb019409447052b to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionWithMonthlyPolicy/f40c877e8a9ed20266673cd1013cd07f/family1/27fd86adcfaf4b5bafb019409447052b 2023-11-28 03:53:48,402 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithMonthlyPolicy/f40c877e8a9ed20266673cd1013cd07f/family1/86811704da1c4131a8c0c0276c5b0489 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionWithMonthlyPolicy/f40c877e8a9ed20266673cd1013cd07f/family1/86811704da1c4131a8c0c0276c5b0489 2023-11-28 03:53:48,404 INFO [RS:0;jenkins-hbase4:41203-shortCompactions-0] throttle.PressureAwareThroughputController(145): f40c877e8a9ed20266673cd1013cd07f#family1#compaction#23 average throughput is 14.15 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-11-28 03:53:48,409 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithMonthlyPolicy/f40c877e8a9ed20266673cd1013cd07f/family1/0b61e1d653d941fca9dac41c5ae432e5 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionWithMonthlyPolicy/f40c877e8a9ed20266673cd1013cd07f/family1/0b61e1d653d941fca9dac41c5ae432e5 2023-11-28 03:53:48,412 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithMonthlyPolicy/f40c877e8a9ed20266673cd1013cd07f/family1/09765f32b2ab48f99943c02962e694bc to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionWithMonthlyPolicy/f40c877e8a9ed20266673cd1013cd07f/family1/09765f32b2ab48f99943c02962e694bc 2023-11-28 03:53:48,415 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithMonthlyPolicy/f40c877e8a9ed20266673cd1013cd07f/family1/d61efffc63eb45a894012197f03f05f0 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionWithMonthlyPolicy/f40c877e8a9ed20266673cd1013cd07f/family1/d61efffc63eb45a894012197f03f05f0 2023-11-28 03:53:48,418 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithMonthlyPolicy/f40c877e8a9ed20266673cd1013cd07f/family1/1c16386b44e8460ebc1722d036f73ddb to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionWithMonthlyPolicy/f40c877e8a9ed20266673cd1013cd07f/family1/1c16386b44e8460ebc1722d036f73ddb 2023-11-28 03:53:48,421 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithMonthlyPolicy/f40c877e8a9ed20266673cd1013cd07f/family1/18c80b53fbbe4f2e9e2c76311af62902_SeqId_44_ to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionWithMonthlyPolicy/f40c877e8a9ed20266673cd1013cd07f/family1/18c80b53fbbe4f2e9e2c76311af62902_SeqId_44_ 2023-11-28 03:53:48,440 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithMonthlyPolicy/f40c877e8a9ed20266673cd1013cd07f/.tmp/family1/09848eba430c44deaba7a47c615727c5 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithMonthlyPolicy/f40c877e8a9ed20266673cd1013cd07f/family1/09848eba430c44deaba7a47c615727c5 2023-11-28 03:53:48,447 INFO [RS:0;jenkins-hbase4:41203-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in f40c877e8a9ed20266673cd1013cd07f/family1 of f40c877e8a9ed20266673cd1013cd07f into 09848eba430c44deaba7a47c615727c5(size=167.4 K), total size for store is 167.4 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-11-28 03:53:48,447 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for f40c877e8a9ed20266673cd1013cd07f: 2023-11-28 03:53:48,447 INFO [RS:0;jenkins-hbase4:41203-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithMonthlyPolicy,,1701143618810.f40c877e8a9ed20266673cd1013cd07f., storeName=family1, fileCount=3, fileSize=178.1 K (166.0 K, 6.6 K, 5.5 K), priority=2, time=1701143628371; duration=0sec 2023-11-28 03:53:48,447 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-28 03:53:48,470 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511301bb158770ff14881bb25ee018805b7ab to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201511301bb158770ff14881bb25ee018805b7ab 2023-11-28 03:53:48,471 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-11-28 03:53:48,481 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithMonthlyPolicy/f40c877e8a9ed20266673cd1013cd07f/family1/05c7e1f03d3b450fa115adc607f67278, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithMonthlyPolicy/f40c877e8a9ed20266673cd1013cd07f/family1/9d94a50280d647a9a92ce19b916a1269, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithMonthlyPolicy/f40c877e8a9ed20266673cd1013cd07f/family1/f546da9f7fd640b6b02aa6c8f72bd361_SeqId_47_] to archive 2023-11-28 03:53:48,482 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-28 03:53:48,484 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithMonthlyPolicy/f40c877e8a9ed20266673cd1013cd07f/family1/05c7e1f03d3b450fa115adc607f67278 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionWithMonthlyPolicy/f40c877e8a9ed20266673cd1013cd07f/family1/05c7e1f03d3b450fa115adc607f67278 2023-11-28 03:53:48,486 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithMonthlyPolicy/f40c877e8a9ed20266673cd1013cd07f/family1/9d94a50280d647a9a92ce19b916a1269 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionWithMonthlyPolicy/f40c877e8a9ed20266673cd1013cd07f/family1/9d94a50280d647a9a92ce19b916a1269 2023-11-28 03:53:48,488 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithMonthlyPolicy/f40c877e8a9ed20266673cd1013cd07f/family1/f546da9f7fd640b6b02aa6c8f72bd361_SeqId_47_ to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionWithMonthlyPolicy/f40c877e8a9ed20266673cd1013cd07f/family1/f546da9f7fd640b6b02aa6c8f72bd361_SeqId_47_ 2023-11-28 03:53:48,500 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/35f12f7266b347ac936f32139415b3cb first=Optional[r00] last=Optional[r8] 2023-11-28 03:53:48,501 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithMonthlyPolicy,,1701143618810.f40c877e8a9ed20266673cd1013cd07f., hostname=jenkins-hbase4.apache.org,41203,1701143600653, seqNum=2 for row with hfile group [{family1,hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/35f12f7266b347ac936f32139415b3cb}] 2023-11-28 03:53:48,504 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HStore(763): Validating hfile at hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/35f12f7266b347ac936f32139415b3cb for inclusion in f40c877e8a9ed20266673cd1013cd07f/family1 2023-11-28 03:53:48,507 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HStore(775): HFile bounds: first=r00 last=r8 2023-11-28 03:53:48,507 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HStore(777): Region bounds: first= last= 2023-11-28 03:53:48,507 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HRegion(2446): Flush status journal for f40c877e8a9ed20266673cd1013cd07f: 2023-11-28 03:53:48,508 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/35f12f7266b347ac936f32139415b3cb to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testMajorCompactionWithMonthlyPolicy__u4g3aa492cgtuaplhoc99jasfr6eggb6ib321c9srm9n6q5britb7h5qg8ruunp5/family1/35f12f7266b347ac936f32139415b3cb 2023-11-28 03:53:48,511 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testMajorCompactionWithMonthlyPolicy__u4g3aa492cgtuaplhoc99jasfr6eggb6ib321c9srm9n6q5britb7h5qg8ruunp5/family1/35f12f7266b347ac936f32139415b3cb as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithMonthlyPolicy/f40c877e8a9ed20266673cd1013cd07f/family1/02a3466179bb43aaabcba4a6087ad123_SeqId_50_ 2023-11-28 03:53:48,512 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HStore(849): Loaded HFile hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testMajorCompactionWithMonthlyPolicy__u4g3aa492cgtuaplhoc99jasfr6eggb6ib321c9srm9n6q5britb7h5qg8ruunp5/family1/35f12f7266b347ac936f32139415b3cb into f40c877e8a9ed20266673cd1013cd07f/family1 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithMonthlyPolicy/f40c877e8a9ed20266673cd1013cd07f/family1/02a3466179bb43aaabcba4a6087ad123_SeqId_50_ - updating store file list. 2023-11-28 03:53:48,519 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HStore(882): Loaded HFile hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithMonthlyPolicy/f40c877e8a9ed20266673cd1013cd07f/family1/02a3466179bb43aaabcba4a6087ad123_SeqId_50_ into f40c877e8a9ed20266673cd1013cd07f/family1 2023-11-28 03:53:48,519 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HStore(855): Successfully loaded hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testMajorCompactionWithMonthlyPolicy__u4g3aa492cgtuaplhoc99jasfr6eggb6ib321c9srm9n6q5britb7h5qg8ruunp5/family1/35f12f7266b347ac936f32139415b3cb into f40c877e8a9ed20266673cd1013cd07f/family1 (new location: hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithMonthlyPolicy/f40c877e8a9ed20266673cd1013cd07f/family1/02a3466179bb43aaabcba4a6087ad123_SeqId_50_) 2023-11-28 03:53:48,519 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testMajorCompactionWithMonthlyPolicy__u4g3aa492cgtuaplhoc99jasfr6eggb6ib321c9srm9n6q5britb7h5qg8ruunp5/family1/35f12f7266b347ac936f32139415b3cb 2023-11-28 03:53:48,522 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-11-28 03:53:48,523 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-28 03:53:48,525 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151120561c856e2f46400bb66e4929aa27d57d to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151120561c856e2f46400bb66e4929aa27d57d 2023-11-28 03:53:48,527 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151128917da52aa1ce4736a7d002f4c93b7844 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151128917da52aa1ce4736a7d002f4c93b7844 2023-11-28 03:53:48,528 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151130dba525280ff54764b654cb630edd6af6 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151130dba525280ff54764b654cb630edd6af6 2023-11-28 03:53:48,528 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithMonthlyPolicy 3 expired mob files are deleted 2023-11-28 03:53:48,528 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101' 2023-11-28 03:53:48,529 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-11-28 03:53:48,529 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-11-28 03:53:48,529 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-11-28 03:53:48,529 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-11-28 03:53:48,529 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithMonthlyPolicy', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-11-28 03:53:48,530 DEBUG [PEWorker-3] locking.LockProcedure(242): UNLOCKED pid=22, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-11-28 03:53:48,532 INFO [PEWorker-3] procedure2.ProcedureExecutor(1409): Finished pid=22, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE in 1.9790 sec 2023-11-28 03:53:48,645 INFO [Listener at localhost/42853] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionWithMonthlyPolicy Thread=260 (was 249) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/2f2fe2ad-19b7-95b4-b267-35017f394f29/cluster_c657509c-448c-2cac-5018-4e5cd83a6dc3/dfs/data/data2/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1016373073_17 at /127.0.0.1:59454 [Waiting for operation #14] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1016373073_17 at /127.0.0.1:59284 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x116f23d0-shared-pool-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1016373073_17 at /127.0.0.1:59222 [Waiting for operation #12] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x116f23d0-shared-pool-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x116f23d0-shared-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1016373073_17 at /127.0.0.1:59444 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1016373073_17 at /127.0.0.1:59420 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-36928057_17 at /127.0.0.1:59354 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1016373073_17 at /127.0.0.1:59356 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1016373073_17 at /127.0.0.1:48720 [Waiting for operation #35] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x116f23d0-metaLookup-shared--pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1016373073_17 at /127.0.0.1:59326 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-36928057_17 at /127.0.0.1:59376 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1016373073_17 at /127.0.0.1:59502 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1016373073_17 at /127.0.0.1:59254 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:55641@0x56d83728-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:549) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/2f2fe2ad-19b7-95b4-b267-35017f394f29/cluster_c657509c-448c-2cac-5018-4e5cd83a6dc3/dfs/data/data1/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:55641@0x56d83728 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$88/1587213957.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1016373073_17 at /127.0.0.1:59296 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x116f23d0-shared-pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1016373073_17 at /127.0.0.1:59340 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1016373073_17 at /127.0.0.1:59300 [Waiting for operation #18] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:55641@0x56d83728-SendThread(127.0.0.1:55641) sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:332) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1289) Potentially hanging thread: hconnection-0x116f23d0-shared-pool-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1016373073_17 at /127.0.0.1:48836 [Waiting for operation #26] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1016373073_17 at /127.0.0.1:59406 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x1ad0a343-shared-pool-23 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x116f23d0-shared-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=644 (was 618) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=259 (was 273), ProcessCount=171 (was 172), AvailableMemoryMB=7128 (was 7149) 2023-11-28 03:53:48,656 INFO [Listener at localhost/42853] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMinorCompactionWithWeeklyPolicy Thread=260, OpenFileDescriptor=644, MaxFileDescriptor=60000, SystemLoadAverage=259, ProcessCount=172, AvailableMemoryMB=7128 2023-11-28 03:53:48,658 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testMinorCompactionWithWeeklyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-11-28 03:53:48,659 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] procedure2.ProcedureExecutor(1028): Stored pid=23, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:48,660 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_PRE_OPERATION 2023-11-28 03:53:48,661 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(700): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMinorCompactionWithWeeklyPolicy" procId is: 23 2023-11-28 03:53:48,661 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-11-28 03:53:48,661 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(1230): Checking to see if procedure is done pid=23 2023-11-28 03:53:48,663 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/.tmp/data/default/testMinorCompactionWithWeeklyPolicy/0758c3f3bb1fb9883b18f53f29d6126b 2023-11-28 03:53:48,664 DEBUG [HFileArchiver-4] backup.HFileArchiver(153): Directory hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/.tmp/data/default/testMinorCompactionWithWeeklyPolicy/0758c3f3bb1fb9883b18f53f29d6126b empty. 2023-11-28 03:53:48,664 DEBUG [HFileArchiver-4] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/.tmp/data/default/testMinorCompactionWithWeeklyPolicy/0758c3f3bb1fb9883b18f53f29d6126b 2023-11-28 03:53:48,665 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(328): Archived testMinorCompactionWithWeeklyPolicy regions 2023-11-28 03:53:48,762 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(1230): Checking to see if procedure is done pid=23 2023-11-28 03:53:48,964 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(1230): Checking to see if procedure is done pid=23 2023-11-28 03:53:49,074 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/.tmp/data/default/testMinorCompactionWithWeeklyPolicy/.tabledesc/.tableinfo.0000000001 2023-11-28 03:53:49,076 INFO [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(7675): creating {ENCODED => 0758c3f3bb1fb9883b18f53f29d6126b, NAME => 'testMinorCompactionWithWeeklyPolicy,,1701143628658.0758c3f3bb1fb9883b18f53f29d6126b.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMinorCompactionWithWeeklyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}, regionDir=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/.tmp 2023-11-28 03:53:49,265 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(1230): Checking to see if procedure is done pid=23 2023-11-28 03:53:49,483 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(866): Instantiated testMinorCompactionWithWeeklyPolicy,,1701143628658.0758c3f3bb1fb9883b18f53f29d6126b.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-28 03:53:49,483 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1604): Closing 0758c3f3bb1fb9883b18f53f29d6126b, disabling compactions & flushes 2023-11-28 03:53:49,483 INFO [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1626): Closing region testMinorCompactionWithWeeklyPolicy,,1701143628658.0758c3f3bb1fb9883b18f53f29d6126b. 2023-11-28 03:53:49,483 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMinorCompactionWithWeeklyPolicy,,1701143628658.0758c3f3bb1fb9883b18f53f29d6126b. 2023-11-28 03:53:49,483 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1714): Acquired close lock on testMinorCompactionWithWeeklyPolicy,,1701143628658.0758c3f3bb1fb9883b18f53f29d6126b. after waiting 0 ms 2023-11-28 03:53:49,483 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1724): Updates disabled for region testMinorCompactionWithWeeklyPolicy,,1701143628658.0758c3f3bb1fb9883b18f53f29d6126b. 2023-11-28 03:53:49,483 INFO [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1838): Closed testMinorCompactionWithWeeklyPolicy,,1701143628658.0758c3f3bb1fb9883b18f53f29d6126b. 2023-11-28 03:53:49,483 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1558): Region close journal for 0758c3f3bb1fb9883b18f53f29d6126b: 2023-11-28 03:53:49,486 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_ADD_TO_META 2023-11-28 03:53:49,487 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMinorCompactionWithWeeklyPolicy,,1701143628658.0758c3f3bb1fb9883b18f53f29d6126b.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1701143629487"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1701143629487"}]},"ts":"1701143629487"} 2023-11-28 03:53:49,489 INFO [PEWorker-4] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-11-28 03:53:49,489 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-11-28 03:53:49,490 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMinorCompactionWithWeeklyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1701143629490"}]},"ts":"1701143629490"} 2023-11-28 03:53:49,491 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testMinorCompactionWithWeeklyPolicy, state=ENABLING in hbase:meta 2023-11-28 03:53:49,494 INFO [PEWorker-4] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=24, ppid=23, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMinorCompactionWithWeeklyPolicy, region=0758c3f3bb1fb9883b18f53f29d6126b, ASSIGN}] 2023-11-28 03:53:49,496 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=24, ppid=23, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMinorCompactionWithWeeklyPolicy, region=0758c3f3bb1fb9883b18f53f29d6126b, ASSIGN 2023-11-28 03:53:49,497 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=24, ppid=23, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMinorCompactionWithWeeklyPolicy, region=0758c3f3bb1fb9883b18f53f29d6126b, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,41203,1701143600653; forceNewPlan=false, retain=false 2023-11-28 03:53:49,648 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=24 updating hbase:meta row=0758c3f3bb1fb9883b18f53f29d6126b, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:49,648 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMinorCompactionWithWeeklyPolicy,,1701143628658.0758c3f3bb1fb9883b18f53f29d6126b.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1701143629648"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1701143629648"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1701143629648"}]},"ts":"1701143629648"} 2023-11-28 03:53:49,650 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=25, ppid=24, state=RUNNABLE; OpenRegionProcedure 0758c3f3bb1fb9883b18f53f29d6126b, server=jenkins-hbase4.apache.org,41203,1701143600653}] 2023-11-28 03:53:49,766 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(1230): Checking to see if procedure is done pid=23 2023-11-28 03:53:49,802 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:49,806 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testMinorCompactionWithWeeklyPolicy,,1701143628658.0758c3f3bb1fb9883b18f53f29d6126b. 2023-11-28 03:53:49,806 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 0758c3f3bb1fb9883b18f53f29d6126b, NAME => 'testMinorCompactionWithWeeklyPolicy,,1701143628658.0758c3f3bb1fb9883b18f53f29d6126b.', STARTKEY => '', ENDKEY => ''} 2023-11-28 03:53:49,806 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMinorCompactionWithWeeklyPolicy 0758c3f3bb1fb9883b18f53f29d6126b 2023-11-28 03:53:49,806 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMinorCompactionWithWeeklyPolicy,,1701143628658.0758c3f3bb1fb9883b18f53f29d6126b.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-28 03:53:49,807 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 0758c3f3bb1fb9883b18f53f29d6126b 2023-11-28 03:53:49,807 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 0758c3f3bb1fb9883b18f53f29d6126b 2023-11-28 03:53:49,808 INFO [StoreOpener-0758c3f3bb1fb9883b18f53f29d6126b-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 0758c3f3bb1fb9883b18f53f29d6126b 2023-11-28 03:53:49,809 DEBUG [StoreOpener-0758c3f3bb1fb9883b18f53f29d6126b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithWeeklyPolicy/0758c3f3bb1fb9883b18f53f29d6126b/family1 2023-11-28 03:53:49,809 DEBUG [StoreOpener-0758c3f3bb1fb9883b18f53f29d6126b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithWeeklyPolicy/0758c3f3bb1fb9883b18f53f29d6126b/family1 2023-11-28 03:53:49,810 INFO [StoreOpener-0758c3f3bb1fb9883b18f53f29d6126b-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 0758c3f3bb1fb9883b18f53f29d6126b columnFamilyName family1 2023-11-28 03:53:49,811 INFO [StoreOpener-0758c3f3bb1fb9883b18f53f29d6126b-1] regionserver.HStore(310): Store=0758c3f3bb1fb9883b18f53f29d6126b/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-28 03:53:49,812 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithWeeklyPolicy/0758c3f3bb1fb9883b18f53f29d6126b 2023-11-28 03:53:49,813 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithWeeklyPolicy/0758c3f3bb1fb9883b18f53f29d6126b 2023-11-28 03:53:49,816 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 0758c3f3bb1fb9883b18f53f29d6126b 2023-11-28 03:53:49,818 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithWeeklyPolicy/0758c3f3bb1fb9883b18f53f29d6126b/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-11-28 03:53:49,818 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 0758c3f3bb1fb9883b18f53f29d6126b; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72158034, jitterRate=0.07523849606513977}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-11-28 03:53:49,818 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 0758c3f3bb1fb9883b18f53f29d6126b: 2023-11-28 03:53:49,819 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2339): Post open deploy tasks for testMinorCompactionWithWeeklyPolicy,,1701143628658.0758c3f3bb1fb9883b18f53f29d6126b., pid=25, masterSystemTime=1701143629802 2023-11-28 03:53:49,821 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2366): Finished post open deploy task for testMinorCompactionWithWeeklyPolicy,,1701143628658.0758c3f3bb1fb9883b18f53f29d6126b. 2023-11-28 03:53:49,821 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testMinorCompactionWithWeeklyPolicy,,1701143628658.0758c3f3bb1fb9883b18f53f29d6126b. 2023-11-28 03:53:49,821 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=24 updating hbase:meta row=0758c3f3bb1fb9883b18f53f29d6126b, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:49,822 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMinorCompactionWithWeeklyPolicy,,1701143628658.0758c3f3bb1fb9883b18f53f29d6126b.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1701143629821"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1701143629821"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1701143629821"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1701143629821"}]},"ts":"1701143629821"} 2023-11-28 03:53:49,825 INFO [PEWorker-3] procedure2.ProcedureExecutor(1825): Finished subprocedure pid=25, resume processing ppid=24 2023-11-28 03:53:49,825 INFO [PEWorker-3] procedure2.ProcedureExecutor(1409): Finished pid=25, ppid=24, state=SUCCESS; OpenRegionProcedure 0758c3f3bb1fb9883b18f53f29d6126b, server=jenkins-hbase4.apache.org,41203,1701143600653 in 173 msec 2023-11-28 03:53:49,827 INFO [PEWorker-4] procedure2.ProcedureExecutor(1825): Finished subprocedure pid=24, resume processing ppid=23 2023-11-28 03:53:49,827 INFO [PEWorker-4] procedure2.ProcedureExecutor(1409): Finished pid=24, ppid=23, state=SUCCESS; TransitRegionStateProcedure table=testMinorCompactionWithWeeklyPolicy, region=0758c3f3bb1fb9883b18f53f29d6126b, ASSIGN in 331 msec 2023-11-28 03:53:49,828 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-11-28 03:53:49,829 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMinorCompactionWithWeeklyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1701143629828"}]},"ts":"1701143629828"} 2023-11-28 03:53:49,830 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testMinorCompactionWithWeeklyPolicy, state=ENABLED in hbase:meta 2023-11-28 03:53:49,832 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_POST_OPERATION 2023-11-28 03:53:49,834 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=23, state=SUCCESS; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy in 1.1740 sec 2023-11-28 03:53:50,767 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(1230): Checking to see if procedure is done pid=23 2023-11-28 03:53:50,768 INFO [Listener at localhost/42853] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMinorCompactionWithWeeklyPolicy, procId: 23 completed 2023-11-28 03:53:50,788 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-28 03:53:50,790 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:50,790 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-11-28 03:53:50,790 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:53:50,790 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-11-28 03:53:50,790 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-11-28 03:53:50,791 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:50,791 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:50,793 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:53:50,793 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:50,793 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:53:50,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:53:50,793 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:50,793 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-28 03:53:50,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:50,794 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:50,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-28 03:53:50,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:50,794 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:50,794 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:50,794 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-11-28 03:53:50,794 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:53:50,794 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-11-28 03:53:50,795 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-28 03:53:50,795 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-28 03:53:50,795 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1701143628658.0758c3f3bb1fb9883b18f53f29d6126b. 2023-11-28 03:53:50,795 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1701143628658.0758c3f3bb1fb9883b18f53f29d6126b. started... 2023-11-28 03:53:50,795 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 0758c3f3bb1fb9883b18f53f29d6126b 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-11-28 03:53:51,215 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151130d17008ffb9124414858a40e18b90b635 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151130d17008ffb9124414858a40e18b90b635 2023-11-28 03:53:51,620 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithWeeklyPolicy/0758c3f3bb1fb9883b18f53f29d6126b/.tmp/family1/0d5417d199954793a2b77cda50419a18 2023-11-28 03:53:51,627 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithWeeklyPolicy/0758c3f3bb1fb9883b18f53f29d6126b/.tmp/family1/0d5417d199954793a2b77cda50419a18 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithWeeklyPolicy/0758c3f3bb1fb9883b18f53f29d6126b/family1/0d5417d199954793a2b77cda50419a18 2023-11-28 03:53:51,632 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithWeeklyPolicy/0758c3f3bb1fb9883b18f53f29d6126b/family1/0d5417d199954793a2b77cda50419a18, entries=1000, sequenceid=5, filesize=165.1 K 2023-11-28 03:53:51,633 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for 0758c3f3bb1fb9883b18f53f29d6126b in 838ms, sequenceid=5, compaction requested=false 2023-11-28 03:53:51,634 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMinorCompactionWithWeeklyPolicy' 2023-11-28 03:53:51,634 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 0758c3f3bb1fb9883b18f53f29d6126b: 2023-11-28 03:53:51,634 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1701143628658.0758c3f3bb1fb9883b18f53f29d6126b. 2023-11-28 03:53:51,634 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-28 03:53:51,635 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-28 03:53:51,635 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:51,635 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-11-28 03:53:51,635 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-11-28 03:53:51,637 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:51,637 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:51,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:51,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:53:51,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:53:51,637 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:51,637 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-28 03:53:51,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:53:51,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:53:51,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:51,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:51,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:53:51,638 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-11-28 03:53:51,638 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@73d3e700[Count = 0] remaining members to acquire global barrier 2023-11-28 03:53:51,638 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-11-28 03:53:51,639 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:51,640 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:51,640 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:51,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:51,640 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-11-28 03:53:51,640 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-11-28 03:53:51,640 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,41203,1701143600653' in zk 2023-11-28 03:53:51,640 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:51,640 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-28 03:53:51,642 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:51,642 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-11-28 03:53:51,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:51,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:53:51,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:53:51,642 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:53:51,642 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-11-28 03:53:51,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:53:51,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:53:51,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:51,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:51,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:53:51,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:51,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:51,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,41203,1701143600653': 2023-11-28 03:53:51,645 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-11-28 03:53:51,645 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-11-28 03:53:51,645 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-28 03:53:51,645 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-28 03:53:51,645 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:51,645 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-28 03:53:51,648 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:51,648 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:51,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:51,648 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:51,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:53:51,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:53:51,648 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:53:51,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:51,649 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:51,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:53:51,649 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:53:51,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:53:51,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:51,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:51,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:53:51,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:51,650 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:51,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:51,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:53:51,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:51,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:51,653 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:51,653 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:53:51,653 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:53:51,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:53:51,653 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:53:51,653 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(399): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-11-28 03:53:51,653 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-11-28 03:53:51,654 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-28 03:53:51,654 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:53:51,654 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:51,654 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:53:51,654 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:51,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:53:51,654 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:51,654 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-11-28 03:53:51,654 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:51,654 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:51,654 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:51,654 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-28 03:53:51,755 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-28 03:53:51,756 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-28 03:53:51,760 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-28 03:53:51,762 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-11-28 03:53:51,762 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:51,762 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-11-28 03:53:51,762 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:53:51,762 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-11-28 03:53:51,762 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-11-28 03:53:51,763 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:51,763 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:51,764 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:51,764 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:53:51,765 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:53:51,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:53:51,765 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:51,765 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-28 03:53:51,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:51,765 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:51,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-28 03:53:51,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:51,766 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:51,766 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-11-28 03:53:51,766 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:51,766 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-11-28 03:53:51,766 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:53:51,766 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-11-28 03:53:51,766 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-28 03:53:51,766 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-28 03:53:51,766 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1701143628658.0758c3f3bb1fb9883b18f53f29d6126b. 2023-11-28 03:53:51,766 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1701143628658.0758c3f3bb1fb9883b18f53f29d6126b. started... 2023-11-28 03:53:51,767 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 0758c3f3bb1fb9883b18f53f29d6126b 1/1 column families, dataSize=74 B heapSize=408 B 2023-11-28 03:53:52,181 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511283676a401a01d41f6a035489886681573 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201511283676a401a01d41f6a035489886681573 2023-11-28 03:53:52,587 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithWeeklyPolicy/0758c3f3bb1fb9883b18f53f29d6126b/.tmp/family1/9683ab0ae7784e849f4f409399289833 2023-11-28 03:53:52,593 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithWeeklyPolicy/0758c3f3bb1fb9883b18f53f29d6126b/.tmp/family1/9683ab0ae7784e849f4f409399289833 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithWeeklyPolicy/0758c3f3bb1fb9883b18f53f29d6126b/family1/9683ab0ae7784e849f4f409399289833 2023-11-28 03:53:52,597 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithWeeklyPolicy/0758c3f3bb1fb9883b18f53f29d6126b/family1/9683ab0ae7784e849f4f409399289833, entries=1, sequenceid=9, filesize=5.0 K 2023-11-28 03:53:52,598 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for 0758c3f3bb1fb9883b18f53f29d6126b in 831ms, sequenceid=9, compaction requested=false 2023-11-28 03:53:52,598 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 0758c3f3bb1fb9883b18f53f29d6126b: 2023-11-28 03:53:52,598 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1701143628658.0758c3f3bb1fb9883b18f53f29d6126b. 2023-11-28 03:53:52,598 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-28 03:53:52,598 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-28 03:53:52,598 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:52,599 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-11-28 03:53:52,599 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-11-28 03:53:52,603 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:52,603 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:52,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:52,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:53:52,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:53:52,604 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:52,604 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-28 03:53:52,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:53:52,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:53:52,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:52,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:52,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:53:52,605 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-11-28 03:53:52,605 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@16045a0c[Count = 0] remaining members to acquire global barrier 2023-11-28 03:53:52,605 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-11-28 03:53:52,605 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:52,607 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:52,607 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:52,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:52,607 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-11-28 03:53:52,607 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-11-28 03:53:52,607 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,41203,1701143600653' in zk 2023-11-28 03:53:52,607 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:52,607 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-28 03:53:52,608 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-11-28 03:53:52,608 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:53:52,608 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:52,608 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-11-28 03:53:52,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:52,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:53:52,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:53:52,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:53:52,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:53:52,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:52,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:52,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:53:52,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:52,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:52,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,41203,1701143600653': 2023-11-28 03:53:52,611 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-11-28 03:53:52,611 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-11-28 03:53:52,611 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-28 03:53:52,611 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-28 03:53:52,611 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:52,611 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-28 03:53:52,613 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:52,613 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:53:52,613 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:52,613 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:52,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:52,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:52,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:53:52,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:53:52,613 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:53:52,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:53:52,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:53:52,614 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:52,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:52,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:52,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:53:52,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:52,615 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:52,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:52,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:53:52,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:52,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:52,618 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:53:52,618 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:53:52,618 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:53:52,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:53:52,618 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:53:52,618 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(399): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-11-28 03:53:52,618 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:52,618 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-11-28 03:53:52,619 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-28 03:53:52,619 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:53:52,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:53:52,619 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:52,619 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:52,619 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:52,619 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-11-28 03:53:52,619 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:52,619 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-28 03:53:52,619 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:52,619 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:52,719 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-28 03:53:52,720 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-28 03:53:52,725 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-28 03:53:52,726 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-11-28 03:53:52,726 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:52,726 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-11-28 03:53:52,726 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:53:52,727 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-11-28 03:53:52,727 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-11-28 03:53:52,727 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:52,727 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:52,728 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:53:52,728 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:52,729 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:53:52,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:53:52,729 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:52,729 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-28 03:53:52,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:52,729 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:52,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-28 03:53:52,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:52,730 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:52,730 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-11-28 03:53:52,730 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:52,730 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-11-28 03:53:52,730 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:53:52,730 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-11-28 03:53:52,730 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-28 03:53:52,730 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-28 03:53:52,730 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1701143628658.0758c3f3bb1fb9883b18f53f29d6126b. 2023-11-28 03:53:52,731 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1701143628658.0758c3f3bb1fb9883b18f53f29d6126b. started... 2023-11-28 03:53:52,731 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 0758c3f3bb1fb9883b18f53f29d6126b 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-28 03:53:53,148 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151201d1de3486eda14e3481c48434f1f6149a to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151201d1de3486eda14e3481c48434f1f6149a 2023-11-28 03:53:53,554 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithWeeklyPolicy/0758c3f3bb1fb9883b18f53f29d6126b/.tmp/family1/1a92fa33465d4ad2bb8e9997f627f073 2023-11-28 03:53:53,560 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithWeeklyPolicy/0758c3f3bb1fb9883b18f53f29d6126b/.tmp/family1/1a92fa33465d4ad2bb8e9997f627f073 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithWeeklyPolicy/0758c3f3bb1fb9883b18f53f29d6126b/family1/1a92fa33465d4ad2bb8e9997f627f073 2023-11-28 03:53:53,565 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithWeeklyPolicy/0758c3f3bb1fb9883b18f53f29d6126b/family1/1a92fa33465d4ad2bb8e9997f627f073, entries=1, sequenceid=13, filesize=4.9 K 2023-11-28 03:53:53,566 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 0758c3f3bb1fb9883b18f53f29d6126b in 835ms, sequenceid=13, compaction requested=true 2023-11-28 03:53:53,566 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 0758c3f3bb1fb9883b18f53f29d6126b: 2023-11-28 03:53:53,566 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1701143628658.0758c3f3bb1fb9883b18f53f29d6126b. 2023-11-28 03:53:53,566 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-28 03:53:53,566 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-28 03:53:53,566 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:53,566 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-11-28 03:53:53,566 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-11-28 03:53:53,568 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:53,568 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:53,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:53,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:53:53,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:53:53,568 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:53,568 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-28 03:53:53,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:53:53,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:53:53,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:53,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:53,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:53:53,570 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-11-28 03:53:53,570 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@73adadb4[Count = 0] remaining members to acquire global barrier 2023-11-28 03:53:53,570 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-11-28 03:53:53,570 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:53,571 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:53,571 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:53,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:53,571 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-11-28 03:53:53,572 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-11-28 03:53:53,572 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,41203,1701143600653' in zk 2023-11-28 03:53:53,572 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:53,572 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-28 03:53:53,573 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:53,573 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-11-28 03:53:53,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:53,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:53:53,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:53:53,573 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:53:53,573 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-11-28 03:53:53,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:53:53,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:53:53,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:53,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:53,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:53:53,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:53,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:53,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,41203,1701143600653': 2023-11-28 03:53:53,576 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-11-28 03:53:53,576 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-11-28 03:53:53,576 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-28 03:53:53,576 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-28 03:53:53,576 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:53,576 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-28 03:53:53,579 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:53,579 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:53,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:53,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:53:53,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:53:53,579 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:53,579 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:53:53,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:53,579 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:53:53,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:53:53,579 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:53,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:53:53,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:53,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:53,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:53:53,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:53,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:53,581 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:53,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:53:53,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:53,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:53,583 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:53,583 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:53,583 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:53,583 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:53:53,583 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:53:53,583 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:53,583 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:53:53,583 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-11-28 03:53:53,584 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-28 03:53:53,583 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:53:53,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:53:53,584 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:53,584 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:53,584 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:53,584 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-11-28 03:53:53,584 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-28 03:53:53,584 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:53:53,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:53:53,684 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-28 03:53:53,685 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-28 03:53:53,690 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-28 03:53:53,691 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-11-28 03:53:53,691 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:53,692 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-11-28 03:53:53,692 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:53:53,692 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-11-28 03:53:53,692 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-11-28 03:53:53,692 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:53,692 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:53,694 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:53:53,694 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:53,694 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:53:53,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:53:53,694 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:53,694 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-28 03:53:53,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:53,694 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:53,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-28 03:53:53,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:53,695 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:53,695 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-11-28 03:53:53,695 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:53,695 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-11-28 03:53:53,695 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:53:53,695 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-11-28 03:53:53,695 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-28 03:53:53,695 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-28 03:53:53,695 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1701143628658.0758c3f3bb1fb9883b18f53f29d6126b. 2023-11-28 03:53:53,696 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1701143628658.0758c3f3bb1fb9883b18f53f29d6126b. started... 2023-11-28 03:53:53,696 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 0758c3f3bb1fb9883b18f53f29d6126b 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-28 03:53:54,112 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015120537144d0f53624873972622c5e11ca323 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e2015120537144d0f53624873972622c5e11ca323 2023-11-28 03:53:54,117 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithWeeklyPolicy/0758c3f3bb1fb9883b18f53f29d6126b/.tmp/family1/124e1430343d49f4af3974cbd5e66b75 2023-11-28 03:53:54,123 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithWeeklyPolicy/0758c3f3bb1fb9883b18f53f29d6126b/.tmp/family1/124e1430343d49f4af3974cbd5e66b75 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithWeeklyPolicy/0758c3f3bb1fb9883b18f53f29d6126b/family1/124e1430343d49f4af3974cbd5e66b75 2023-11-28 03:53:54,128 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithWeeklyPolicy/0758c3f3bb1fb9883b18f53f29d6126b/family1/124e1430343d49f4af3974cbd5e66b75, entries=1, sequenceid=17, filesize=4.9 K 2023-11-28 03:53:54,129 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 0758c3f3bb1fb9883b18f53f29d6126b in 433ms, sequenceid=17, compaction requested=true 2023-11-28 03:53:54,129 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 0758c3f3bb1fb9883b18f53f29d6126b: 2023-11-28 03:53:54,130 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1701143628658.0758c3f3bb1fb9883b18f53f29d6126b. 2023-11-28 03:53:54,130 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-28 03:53:54,130 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-28 03:53:54,130 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:54,130 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-11-28 03:53:54,130 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-11-28 03:53:54,132 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:54,132 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:54,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:54,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:53:54,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:53:54,132 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:54,132 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-28 03:53:54,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:53:54,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:53:54,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:54,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:54,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:53:54,134 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-11-28 03:53:54,134 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@8176be0[Count = 0] remaining members to acquire global barrier 2023-11-28 03:53:54,134 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-11-28 03:53:54,135 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:54,137 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:54,137 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:54,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:54,137 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-11-28 03:53:54,137 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-11-28 03:53:54,137 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,41203,1701143600653' in zk 2023-11-28 03:53:54,137 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:54,137 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-28 03:53:54,139 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:54,139 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-11-28 03:53:54,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:54,139 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:53:54,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:53:54,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:53:54,139 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-11-28 03:53:54,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:53:54,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:53:54,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:54,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:54,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:53:54,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:54,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:54,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,41203,1701143600653': 2023-11-28 03:53:54,142 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-11-28 03:53:54,142 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-11-28 03:53:54,142 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-28 03:53:54,143 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-28 03:53:54,143 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:54,143 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-28 03:53:54,144 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:54,144 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:54,145 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:54,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:54,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:53:54,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:53:54,145 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:53:54,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:54,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:53:54,145 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:53:54,145 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:54,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:53:54,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:54,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:54,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:53:54,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:54,147 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:54,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:54,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:53:54,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:54,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:54,154 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:54,154 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(399): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-11-28 03:53:54,154 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-11-28 03:53:54,155 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-28 03:53:54,154 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:53:54,154 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:53:54,155 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:54,155 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:53:54,155 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:53:54,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:53:54,155 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:54,156 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-11-28 03:53:54,156 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:54,156 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-28 03:53:54,156 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:54,156 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:53:54,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:53:54,156 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:54,156 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:54,256 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-28 03:53:54,257 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-28 03:53:54,262 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-28 03:53:54,264 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-11-28 03:53:54,264 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:54,264 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-11-28 03:53:54,264 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:53:54,264 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-11-28 03:53:54,264 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-11-28 03:53:54,265 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:54,265 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:54,266 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:54,266 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:53:54,267 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:53:54,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:53:54,267 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:54,267 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-28 03:53:54,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:54,267 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:54,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-28 03:53:54,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:54,268 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:54,268 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-11-28 03:53:54,268 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:54,268 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-11-28 03:53:54,268 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:53:54,268 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-11-28 03:53:54,269 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-28 03:53:54,269 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-28 03:53:54,269 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1701143628658.0758c3f3bb1fb9883b18f53f29d6126b. 2023-11-28 03:53:54,269 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1701143628658.0758c3f3bb1fb9883b18f53f29d6126b. started... 2023-11-28 03:53:54,269 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 0758c3f3bb1fb9883b18f53f29d6126b 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-28 03:53:54,684 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015122826ed36cdc1ad40a9839d9497ecd27a58 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e2015122826ed36cdc1ad40a9839d9497ecd27a58 2023-11-28 03:53:55,091 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithWeeklyPolicy/0758c3f3bb1fb9883b18f53f29d6126b/.tmp/family1/c0203ba0be3a47bb9e3e061a0daa17e0 2023-11-28 03:53:55,097 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithWeeklyPolicy/0758c3f3bb1fb9883b18f53f29d6126b/.tmp/family1/c0203ba0be3a47bb9e3e061a0daa17e0 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithWeeklyPolicy/0758c3f3bb1fb9883b18f53f29d6126b/family1/c0203ba0be3a47bb9e3e061a0daa17e0 2023-11-28 03:53:55,102 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithWeeklyPolicy/0758c3f3bb1fb9883b18f53f29d6126b/family1/c0203ba0be3a47bb9e3e061a0daa17e0, entries=1, sequenceid=21, filesize=4.9 K 2023-11-28 03:53:55,103 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 0758c3f3bb1fb9883b18f53f29d6126b in 834ms, sequenceid=21, compaction requested=true 2023-11-28 03:53:55,104 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 0758c3f3bb1fb9883b18f53f29d6126b: 2023-11-28 03:53:55,104 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1701143628658.0758c3f3bb1fb9883b18f53f29d6126b. 2023-11-28 03:53:55,104 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-28 03:53:55,104 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-28 03:53:55,104 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:55,104 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-11-28 03:53:55,104 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-11-28 03:53:55,106 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:55,106 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:55,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:55,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:53:55,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:53:55,106 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:55,107 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-28 03:53:55,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:53:55,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:53:55,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:55,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:55,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:53:55,108 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-11-28 03:53:55,108 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@64106605[Count = 0] remaining members to acquire global barrier 2023-11-28 03:53:55,108 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-11-28 03:53:55,108 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:55,110 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:55,110 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:55,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:55,110 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-11-28 03:53:55,110 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-11-28 03:53:55,110 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:55,110 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-28 03:53:55,110 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,41203,1701143600653' in zk 2023-11-28 03:53:55,113 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:55,113 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-11-28 03:53:55,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:55,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:53:55,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:53:55,113 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:53:55,113 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-11-28 03:53:55,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:53:55,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:53:55,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:55,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:55,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:53:55,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:55,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:55,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,41203,1701143600653': 2023-11-28 03:53:55,116 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-11-28 03:53:55,116 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-11-28 03:53:55,116 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-28 03:53:55,116 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-28 03:53:55,116 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:55,116 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-28 03:53:55,118 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:55,118 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:55,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:55,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:53:55,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:53:55,118 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:55,118 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:53:55,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:55,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:53:55,118 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:53:55,118 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:55,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:53:55,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:55,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:55,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:53:55,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:55,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:55,120 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:55,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:53:55,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:55,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:55,122 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:55,122 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:53:55,122 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:55,122 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:53:55,122 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-11-28 03:53:55,123 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-28 03:53:55,122 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:55,122 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:53:55,122 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:53:55,123 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:55,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:53:55,123 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:55,123 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-11-28 03:53:55,123 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:55,123 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-28 03:53:55,123 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:53:55,123 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:55,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:53:55,223 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-28 03:53:55,225 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-28 03:53:55,230 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-28 03:53:55,232 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-11-28 03:53:55,232 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:55,232 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-11-28 03:53:55,232 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:53:55,232 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-11-28 03:53:55,232 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-11-28 03:53:55,233 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:55,233 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:55,234 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:55,234 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:53:55,234 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:53:55,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:53:55,235 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:55,235 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-28 03:53:55,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:55,235 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:55,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-28 03:53:55,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:55,236 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:55,236 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-11-28 03:53:55,236 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:55,236 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-11-28 03:53:55,236 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:53:55,236 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-11-28 03:53:55,236 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-28 03:53:55,236 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-28 03:53:55,236 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1701143628658.0758c3f3bb1fb9883b18f53f29d6126b. 2023-11-28 03:53:55,236 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1701143628658.0758c3f3bb1fb9883b18f53f29d6126b. started... 2023-11-28 03:53:55,236 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 0758c3f3bb1fb9883b18f53f29d6126b 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-28 03:53:55,347 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-11-28 03:53:55,652 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015123183c771a5ce904cc48aad36bde7968168 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e2015123183c771a5ce904cc48aad36bde7968168 2023-11-28 03:53:56,058 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithWeeklyPolicy/0758c3f3bb1fb9883b18f53f29d6126b/.tmp/family1/3eca297c45f2403fa8dec972df370782 2023-11-28 03:53:56,064 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithWeeklyPolicy/0758c3f3bb1fb9883b18f53f29d6126b/.tmp/family1/3eca297c45f2403fa8dec972df370782 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithWeeklyPolicy/0758c3f3bb1fb9883b18f53f29d6126b/family1/3eca297c45f2403fa8dec972df370782 2023-11-28 03:53:56,069 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithWeeklyPolicy/0758c3f3bb1fb9883b18f53f29d6126b/family1/3eca297c45f2403fa8dec972df370782, entries=1, sequenceid=25, filesize=4.9 K 2023-11-28 03:53:56,070 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 0758c3f3bb1fb9883b18f53f29d6126b in 834ms, sequenceid=25, compaction requested=true 2023-11-28 03:53:56,070 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 0758c3f3bb1fb9883b18f53f29d6126b: 2023-11-28 03:53:56,070 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1701143628658.0758c3f3bb1fb9883b18f53f29d6126b. 2023-11-28 03:53:56,070 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-28 03:53:56,070 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-28 03:53:56,070 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:56,070 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-11-28 03:53:56,070 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-11-28 03:53:56,073 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:56,073 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:56,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:56,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:53:56,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:53:56,073 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:56,073 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-28 03:53:56,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:53:56,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:53:56,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:56,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:56,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:53:56,075 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-11-28 03:53:56,075 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@507094b1[Count = 0] remaining members to acquire global barrier 2023-11-28 03:53:56,075 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-11-28 03:53:56,075 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:56,077 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:56,077 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:56,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:56,077 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-11-28 03:53:56,077 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-11-28 03:53:56,077 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,41203,1701143600653' in zk 2023-11-28 03:53:56,077 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:56,077 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-28 03:53:56,079 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-11-28 03:53:56,079 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:56,079 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:53:56,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:56,079 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-11-28 03:53:56,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:53:56,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:53:56,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:53:56,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:53:56,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:56,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:56,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:53:56,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:56,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:56,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,41203,1701143600653': 2023-11-28 03:53:56,082 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-11-28 03:53:56,082 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-11-28 03:53:56,082 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-28 03:53:56,082 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-28 03:53:56,082 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:56,082 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-28 03:53:56,084 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:56,084 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:56,084 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:53:56,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:56,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:53:56,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:53:56,084 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:56,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:56,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:53:56,084 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:53:56,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:53:56,084 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:56,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:56,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:56,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:53:56,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:56,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:56,086 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:56,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:53:56,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:56,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:56,094 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:56,094 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:53:56,095 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:56,095 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-11-28 03:53:56,095 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-28 03:53:56,095 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:53:56,095 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:53:56,095 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:53:56,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:53:56,095 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:56,095 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-11-28 03:53:56,095 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:56,095 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-28 03:53:56,095 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:53:56,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:53:56,095 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:56,096 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:56,096 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:56,195 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-28 03:53:56,197 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-28 03:53:56,201 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-28 03:53:56,203 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-11-28 03:53:56,203 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:56,203 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-11-28 03:53:56,203 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:53:56,203 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-11-28 03:53:56,203 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-11-28 03:53:56,203 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:56,204 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:56,205 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:53:56,205 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:56,205 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:53:56,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:53:56,205 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:56,205 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-28 03:53:56,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:56,206 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:56,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-28 03:53:56,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:56,206 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:56,207 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-11-28 03:53:56,207 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:56,207 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-11-28 03:53:56,207 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:53:56,207 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-11-28 03:53:56,207 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-28 03:53:56,207 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-28 03:53:56,207 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1701143628658.0758c3f3bb1fb9883b18f53f29d6126b. 2023-11-28 03:53:56,207 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1701143628658.0758c3f3bb1fb9883b18f53f29d6126b. started... 2023-11-28 03:53:56,207 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 0758c3f3bb1fb9883b18f53f29d6126b 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-28 03:53:56,624 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2016010164d1f6ef7f0d40b7a753e00386bb8a0f to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e2016010164d1f6ef7f0d40b7a753e00386bb8a0f 2023-11-28 03:53:57,030 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithWeeklyPolicy/0758c3f3bb1fb9883b18f53f29d6126b/.tmp/family1/af2fd9dd748e43e48dbbf3bbce516769 2023-11-28 03:53:57,036 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithWeeklyPolicy/0758c3f3bb1fb9883b18f53f29d6126b/.tmp/family1/af2fd9dd748e43e48dbbf3bbce516769 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithWeeklyPolicy/0758c3f3bb1fb9883b18f53f29d6126b/family1/af2fd9dd748e43e48dbbf3bbce516769 2023-11-28 03:53:57,042 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithWeeklyPolicy/0758c3f3bb1fb9883b18f53f29d6126b/family1/af2fd9dd748e43e48dbbf3bbce516769, entries=1, sequenceid=29, filesize=4.9 K 2023-11-28 03:53:57,043 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 0758c3f3bb1fb9883b18f53f29d6126b in 836ms, sequenceid=29, compaction requested=true 2023-11-28 03:53:57,043 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 0758c3f3bb1fb9883b18f53f29d6126b: 2023-11-28 03:53:57,043 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1701143628658.0758c3f3bb1fb9883b18f53f29d6126b. 2023-11-28 03:53:57,043 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-28 03:53:57,043 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-28 03:53:57,043 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:57,043 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-11-28 03:53:57,043 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-11-28 03:53:57,045 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:57,045 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:57,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:57,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:53:57,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:53:57,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:53:57,045 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:57,045 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-28 03:53:57,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:53:57,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:57,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:57,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:53:57,047 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-11-28 03:53:57,047 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4eae5306[Count = 0] remaining members to acquire global barrier 2023-11-28 03:53:57,047 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-11-28 03:53:57,047 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:57,048 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:57,048 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:57,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:57,048 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-11-28 03:53:57,048 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-11-28 03:53:57,049 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,41203,1701143600653' in zk 2023-11-28 03:53:57,049 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:57,049 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-28 03:53:57,054 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-11-28 03:53:57,054 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:57,054 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:53:57,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:57,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:53:57,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:53:57,055 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-11-28 03:53:57,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:53:57,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:53:57,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:57,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:57,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:53:57,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:57,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:57,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,41203,1701143600653': 2023-11-28 03:53:57,057 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-11-28 03:53:57,057 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-11-28 03:53:57,057 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-28 03:53:57,057 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-28 03:53:57,057 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:57,057 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-28 03:53:57,062 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:57,062 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:57,062 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:57,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:57,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:57,062 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:53:57,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:53:57,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:53:57,062 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:53:57,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:53:57,062 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:57,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:53:57,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:57,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:57,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:53:57,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:57,063 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:57,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:57,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:53:57,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:57,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:57,066 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:57,066 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:57,066 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:57,066 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:57,066 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:57,066 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:57,066 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:57,066 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:53:57,066 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:53:57,066 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(399): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-11-28 03:53:57,066 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-11-28 03:53:57,067 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-28 03:53:57,067 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:53:57,067 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:53:57,067 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-11-28 03:53:57,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:53:57,067 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-28 03:53:57,067 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:53:57,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:53:57,167 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-28 03:53:57,168 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-28 03:53:57,173 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-28 03:53:57,174 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-11-28 03:53:57,174 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:57,174 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-11-28 03:53:57,175 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:53:57,175 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-11-28 03:53:57,175 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-11-28 03:53:57,175 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:57,175 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:57,177 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:57,177 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:53:57,177 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:53:57,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:53:57,178 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:57,178 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-28 03:53:57,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:57,178 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:57,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-28 03:53:57,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:57,179 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:57,179 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-11-28 03:53:57,179 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:57,179 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-11-28 03:53:57,179 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:53:57,179 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-11-28 03:53:57,179 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-28 03:53:57,179 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-28 03:53:57,179 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1701143628658.0758c3f3bb1fb9883b18f53f29d6126b. 2023-11-28 03:53:57,179 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1701143628658.0758c3f3bb1fb9883b18f53f29d6126b. started... 2023-11-28 03:53:57,179 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 0758c3f3bb1fb9883b18f53f29d6126b 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-28 03:53:57,594 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103e83e5105757043849b95ce49688bb079 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20160103e83e5105757043849b95ce49688bb079 2023-11-28 03:53:58,000 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithWeeklyPolicy/0758c3f3bb1fb9883b18f53f29d6126b/.tmp/family1/7e93f5bb599c43598dcba71999291981 2023-11-28 03:53:58,005 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithWeeklyPolicy/0758c3f3bb1fb9883b18f53f29d6126b/.tmp/family1/7e93f5bb599c43598dcba71999291981 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithWeeklyPolicy/0758c3f3bb1fb9883b18f53f29d6126b/family1/7e93f5bb599c43598dcba71999291981 2023-11-28 03:53:58,010 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithWeeklyPolicy/0758c3f3bb1fb9883b18f53f29d6126b/family1/7e93f5bb599c43598dcba71999291981, entries=1, sequenceid=33, filesize=4.9 K 2023-11-28 03:53:58,011 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 0758c3f3bb1fb9883b18f53f29d6126b in 832ms, sequenceid=33, compaction requested=true 2023-11-28 03:53:58,011 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 0758c3f3bb1fb9883b18f53f29d6126b: 2023-11-28 03:53:58,011 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1701143628658.0758c3f3bb1fb9883b18f53f29d6126b. 2023-11-28 03:53:58,011 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-28 03:53:58,011 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-28 03:53:58,011 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:58,011 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-11-28 03:53:58,011 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-11-28 03:53:58,013 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:58,013 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:58,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:58,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:53:58,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:53:58,013 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:58,013 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-28 03:53:58,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:53:58,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:53:58,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:58,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:58,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:53:58,015 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-11-28 03:53:58,015 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7d557958[Count = 0] remaining members to acquire global barrier 2023-11-28 03:53:58,015 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-11-28 03:53:58,015 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:58,016 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:58,016 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:58,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:58,017 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-11-28 03:53:58,017 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-11-28 03:53:58,017 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,41203,1701143600653' in zk 2023-11-28 03:53:58,017 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:58,017 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-28 03:53:58,018 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:58,018 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-11-28 03:53:58,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:58,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:53:58,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:53:58,018 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:53:58,018 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-11-28 03:53:58,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:53:58,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:53:58,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:58,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:58,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:53:58,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:58,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:58,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,41203,1701143600653': 2023-11-28 03:53:58,021 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-11-28 03:53:58,021 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-11-28 03:53:58,021 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-28 03:53:58,021 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-28 03:53:58,021 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:58,021 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-28 03:53:58,023 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:58,023 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:53:58,023 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:58,023 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:58,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:58,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:53:58,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:53:58,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:58,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:53:58,024 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:53:58,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:53:58,024 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:58,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:58,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:58,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:53:58,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:58,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:58,025 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:58,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:53:58,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:58,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:58,028 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:58,028 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:58,028 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:53:58,028 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:58,028 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:53:58,028 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:53:58,028 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:53:58,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:53:58,028 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-11-28 03:53:58,028 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-28 03:53:58,028 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:58,028 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:58,028 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:58,028 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:58,029 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:53:58,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:53:58,029 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-11-28 03:53:58,029 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-28 03:53:58,129 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-28 03:53:58,130 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-28 03:53:58,134 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-28 03:53:58,136 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-11-28 03:53:58,136 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:58,136 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-11-28 03:53:58,136 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:53:58,136 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-11-28 03:53:58,136 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-11-28 03:53:58,136 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:58,136 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:58,138 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:58,138 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:53:58,138 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:53:58,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:53:58,138 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:58,138 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-28 03:53:58,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:58,139 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:58,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-28 03:53:58,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:58,139 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:58,139 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-11-28 03:53:58,139 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:58,139 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-11-28 03:53:58,139 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:53:58,139 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-11-28 03:53:58,140 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-28 03:53:58,140 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-28 03:53:58,140 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1701143628658.0758c3f3bb1fb9883b18f53f29d6126b. 2023-11-28 03:53:58,140 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1701143628658.0758c3f3bb1fb9883b18f53f29d6126b. started... 2023-11-28 03:53:58,140 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 0758c3f3bb1fb9883b18f53f29d6126b 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-28 03:53:58,554 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20150907f5c793f4aced4d4e9fd94d2a0b9488e3 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20150907f5c793f4aced4d4e9fd94d2a0b9488e3 2023-11-28 03:53:58,960 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithWeeklyPolicy/0758c3f3bb1fb9883b18f53f29d6126b/.tmp/family1/e6cd24e717294644b7b7edafac286b07 2023-11-28 03:53:58,966 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithWeeklyPolicy/0758c3f3bb1fb9883b18f53f29d6126b/.tmp/family1/e6cd24e717294644b7b7edafac286b07 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithWeeklyPolicy/0758c3f3bb1fb9883b18f53f29d6126b/family1/e6cd24e717294644b7b7edafac286b07 2023-11-28 03:53:58,971 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithWeeklyPolicy/0758c3f3bb1fb9883b18f53f29d6126b/family1/e6cd24e717294644b7b7edafac286b07, entries=1, sequenceid=37, filesize=4.9 K 2023-11-28 03:53:58,973 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 0758c3f3bb1fb9883b18f53f29d6126b in 833ms, sequenceid=37, compaction requested=true 2023-11-28 03:53:58,973 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 0758c3f3bb1fb9883b18f53f29d6126b: 2023-11-28 03:53:58,973 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1701143628658.0758c3f3bb1fb9883b18f53f29d6126b. 2023-11-28 03:53:58,973 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-28 03:53:58,973 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-28 03:53:58,973 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:58,973 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-11-28 03:53:58,973 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-11-28 03:53:58,975 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:58,975 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:58,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:58,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:53:58,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:53:58,976 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:58,976 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-28 03:53:58,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:53:58,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:53:58,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:58,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:58,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:53:58,977 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-11-28 03:53:58,977 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-11-28 03:53:58,977 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:58,977 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1a1208b9[Count = 0] remaining members to acquire global barrier 2023-11-28 03:53:58,980 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:58,980 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:58,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:58,980 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-11-28 03:53:58,980 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-11-28 03:53:58,980 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,41203,1701143600653' in zk 2023-11-28 03:53:58,980 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:58,980 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-28 03:53:58,981 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:58,981 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-11-28 03:53:58,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:58,982 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:53:58,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:53:58,982 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-11-28 03:53:58,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:53:58,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:53:58,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:53:58,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:58,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:58,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:53:58,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:58,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:58,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,41203,1701143600653': 2023-11-28 03:53:58,985 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-11-28 03:53:58,985 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-11-28 03:53:58,985 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-28 03:53:58,985 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-28 03:53:58,985 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:58,985 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-28 03:53:58,986 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:58,986 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:58,987 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:58,987 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:53:58,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:58,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:58,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:53:58,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:53:58,987 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:53:58,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:53:58,987 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:58,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:53:58,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:58,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:58,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:53:58,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:58,989 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:58,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:58,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:53:58,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:58,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:58,992 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:58,992 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:53:58,992 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:58,992 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:58,992 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:53:58,992 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:53:58,992 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:58,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:53:58,992 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:58,993 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:58,993 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:58,993 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:53:58,993 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:53:58,993 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-11-28 03:53:58,993 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(399): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-11-28 03:53:58,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:53:58,993 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-28 03:53:58,993 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-11-28 03:53:58,993 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-28 03:53:59,094 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-28 03:53:59,095 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-28 03:53:59,099 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-28 03:53:59,101 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-11-28 03:53:59,101 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:59,101 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-11-28 03:53:59,101 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:53:59,101 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-11-28 03:53:59,101 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-11-28 03:53:59,102 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:59,102 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:59,104 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:59,104 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:53:59,104 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:53:59,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:53:59,105 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:59,105 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-28 03:53:59,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:59,105 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:59,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-28 03:53:59,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:59,105 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:59,106 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-11-28 03:53:59,106 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:59,106 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-11-28 03:53:59,106 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:53:59,106 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-11-28 03:53:59,106 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-28 03:53:59,106 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-28 03:53:59,106 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1701143628658.0758c3f3bb1fb9883b18f53f29d6126b. 2023-11-28 03:53:59,106 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1701143628658.0758c3f3bb1fb9883b18f53f29d6126b. started... 2023-11-28 03:53:59,106 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 0758c3f3bb1fb9883b18f53f29d6126b 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-28 03:53:59,521 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151120ad8ff23884f34315a060d4f0fc78bef2 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151120ad8ff23884f34315a060d4f0fc78bef2 2023-11-28 03:53:59,527 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithWeeklyPolicy/0758c3f3bb1fb9883b18f53f29d6126b/.tmp/family1/52ae76ef2c9747f8a8284b172db8f528 2023-11-28 03:53:59,533 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithWeeklyPolicy/0758c3f3bb1fb9883b18f53f29d6126b/.tmp/family1/52ae76ef2c9747f8a8284b172db8f528 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithWeeklyPolicy/0758c3f3bb1fb9883b18f53f29d6126b/family1/52ae76ef2c9747f8a8284b172db8f528 2023-11-28 03:53:59,538 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithWeeklyPolicy/0758c3f3bb1fb9883b18f53f29d6126b/family1/52ae76ef2c9747f8a8284b172db8f528, entries=1, sequenceid=41, filesize=4.9 K 2023-11-28 03:53:59,539 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 0758c3f3bb1fb9883b18f53f29d6126b in 433ms, sequenceid=41, compaction requested=true 2023-11-28 03:53:59,539 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 0758c3f3bb1fb9883b18f53f29d6126b: 2023-11-28 03:53:59,539 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1701143628658.0758c3f3bb1fb9883b18f53f29d6126b. 2023-11-28 03:53:59,539 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-28 03:53:59,539 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-28 03:53:59,539 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:59,539 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-11-28 03:53:59,539 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-11-28 03:53:59,541 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:59,541 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:59,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:59,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:53:59,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:53:59,541 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:59,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:53:59,541 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-28 03:53:59,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:53:59,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:59,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:59,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:53:59,543 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-11-28 03:53:59,543 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4e13d256[Count = 0] remaining members to acquire global barrier 2023-11-28 03:53:59,543 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-11-28 03:53:59,543 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:59,547 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:59,547 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:59,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:59,547 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-11-28 03:53:59,547 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-11-28 03:53:59,547 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:59,547 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-28 03:53:59,547 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,41203,1701143600653' in zk 2023-11-28 03:53:59,550 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:59,550 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-11-28 03:53:59,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:59,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:53:59,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:53:59,550 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:53:59,550 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-11-28 03:53:59,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:53:59,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:53:59,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:59,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:59,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:53:59,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:59,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:59,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,41203,1701143600653': 2023-11-28 03:53:59,553 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-11-28 03:53:59,553 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-11-28 03:53:59,553 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-28 03:53:59,553 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-28 03:53:59,553 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:59,553 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-28 03:53:59,554 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:59,554 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:59,554 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:53:59,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:59,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:53:59,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:53:59,554 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:59,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:59,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:53:59,555 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:53:59,555 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:59,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:53:59,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:59,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:59,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:53:59,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:59,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:59,556 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:59,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:53:59,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:59,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:59,559 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:59,559 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:59,559 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:53:59,559 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:59,559 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:53:59,559 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:53:59,559 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:53:59,559 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-11-28 03:53:59,559 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-28 03:53:59,559 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:53:59,559 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:59,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:53:59,559 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:59,560 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:59,560 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-11-28 03:53:59,560 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-28 03:53:59,560 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:53:59,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:53:59,660 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-28 03:53:59,661 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-28 03:53:59,663 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=38593] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMinorCompactionWithWeeklyPolicy 2023-11-28 03:53:59,665 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] procedure2.ProcedureExecutor(1028): Stored pid=26, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-11-28 03:53:59,665 DEBUG [PEWorker-1] locking.LockProcedure(309): LOCKED pid=26, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-11-28 03:53:59,666 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] mob.MobUtils(833): start MOB compaction of files for table='testMinorCompactionWithWeeklyPolicy', column='family1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-11-28 03:53:59,666 INFO [PEWorker-1] procedure2.TimeoutExecutorThread(81): ADDED pid=26, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1701144239666 2023-11-28 03:53:59,669 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-11-28 03:53:59,670 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] compactions.PartitionedMobCompactor(306): The compaction type is PART_FILES, the request has 0 del files, 9 selected files, and 0 irrelevant files table 'testMinorCompactionWithWeeklyPolicy' and column 'family1' 2023-11-28 03:53:59,671 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-11-28 03:53:59,672 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] zookeeper.ReadOnlyZKClient(139): Connect 0x0329bf50 to 127.0.0.1:55641 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-11-28 03:53:59,676 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] ipc.AbstractRpcClient(189): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@6bb07d46, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-11-28 03:53:59,676 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-11-28 03:54:00,544 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103f4fd69a8d6374793995d76c4e7147ea0 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20160103f4fd69a8d6374793995d76c4e7147ea0 2023-11-28 03:54:00,545 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-11-28 03:54:00,559 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(122): Using SIMPLE authentication for service=ClientService, sasl=false 2023-11-28 03:54:00,561 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:33402, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-11-28 03:54:00,567 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(122): Using SIMPLE authentication for service=MasterService, sasl=false 2023-11-28 03:54:00,569 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:35352, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-11-28 03:54:00,578 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/3d406ecc9cae423dbe2fdc1f1ed7fd44 first=Optional[r3] last=Optional[r6] 2023-11-28 03:54:00,578 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMinorCompactionWithWeeklyPolicy,,1701143628658.0758c3f3bb1fb9883b18f53f29d6126b., hostname=jenkins-hbase4.apache.org,41203,1701143600653, seqNum=2 for row with hfile group [{family1,hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/3d406ecc9cae423dbe2fdc1f1ed7fd44}] 2023-11-28 03:54:00,582 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HStore(763): Validating hfile at hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/3d406ecc9cae423dbe2fdc1f1ed7fd44 for inclusion in 0758c3f3bb1fb9883b18f53f29d6126b/family1 2023-11-28 03:54:00,585 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HStore(775): HFile bounds: first=r3 last=r6 2023-11-28 03:54:00,585 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HStore(777): Region bounds: first= last= 2023-11-28 03:54:00,585 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HRegion(2446): Flush status journal for 0758c3f3bb1fb9883b18f53f29d6126b: 2023-11-28 03:54:00,587 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/3d406ecc9cae423dbe2fdc1f1ed7fd44 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testMinorCompactionWithWeeklyPolicy__b0grj6n2aun7h9o5a2ir5em0f9aft3ldhdnguh8l3licjvslst7b7rj0ithohr2g/family1/3d406ecc9cae423dbe2fdc1f1ed7fd44 2023-11-28 03:54:00,590 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testMinorCompactionWithWeeklyPolicy__b0grj6n2aun7h9o5a2ir5em0f9aft3ldhdnguh8l3licjvslst7b7rj0ithohr2g/family1/3d406ecc9cae423dbe2fdc1f1ed7fd44 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithWeeklyPolicy/0758c3f3bb1fb9883b18f53f29d6126b/family1/ad3a32d354074bc6bf6bab1d686a143d_SeqId_44_ 2023-11-28 03:54:00,590 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HStore(849): Loaded HFile hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testMinorCompactionWithWeeklyPolicy__b0grj6n2aun7h9o5a2ir5em0f9aft3ldhdnguh8l3licjvslst7b7rj0ithohr2g/family1/3d406ecc9cae423dbe2fdc1f1ed7fd44 into 0758c3f3bb1fb9883b18f53f29d6126b/family1 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithWeeklyPolicy/0758c3f3bb1fb9883b18f53f29d6126b/family1/ad3a32d354074bc6bf6bab1d686a143d_SeqId_44_ - updating store file list. 2023-11-28 03:54:00,595 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HStore(882): Loaded HFile hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithWeeklyPolicy/0758c3f3bb1fb9883b18f53f29d6126b/family1/ad3a32d354074bc6bf6bab1d686a143d_SeqId_44_ into 0758c3f3bb1fb9883b18f53f29d6126b/family1 2023-11-28 03:54:00,595 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HStore(855): Successfully loaded hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testMinorCompactionWithWeeklyPolicy__b0grj6n2aun7h9o5a2ir5em0f9aft3ldhdnguh8l3licjvslst7b7rj0ithohr2g/family1/3d406ecc9cae423dbe2fdc1f1ed7fd44 into 0758c3f3bb1fb9883b18f53f29d6126b/family1 (new location: hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithWeeklyPolicy/0758c3f3bb1fb9883b18f53f29d6126b/family1/ad3a32d354074bc6bf6bab1d686a143d_SeqId_44_) 2023-11-28 03:54:00,596 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testMinorCompactionWithWeeklyPolicy__b0grj6n2aun7h9o5a2ir5em0f9aft3ldhdnguh8l3licjvslst7b7rj0ithohr2g/family1/3d406ecc9cae423dbe2fdc1f1ed7fd44 2023-11-28 03:54:00,596 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-11-28 03:54:00,602 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 51214 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-11-28 03:54:00,602 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HStore(1912): 0758c3f3bb1fb9883b18f53f29d6126b/family1 is initiating minor compaction 2023-11-28 03:54:00,602 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@15aef04a; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-28 03:54:00,602 INFO [RS:0;jenkins-hbase4:41203-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 0758c3f3bb1fb9883b18f53f29d6126b/family1 in testMinorCompactionWithWeeklyPolicy,,1701143628658.0758c3f3bb1fb9883b18f53f29d6126b. 2023-11-28 03:54:00,602 INFO [RS:0;jenkins-hbase4:41203-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithWeeklyPolicy/0758c3f3bb1fb9883b18f53f29d6126b/family1/9683ab0ae7784e849f4f409399289833, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithWeeklyPolicy/0758c3f3bb1fb9883b18f53f29d6126b/family1/1a92fa33465d4ad2bb8e9997f627f073, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithWeeklyPolicy/0758c3f3bb1fb9883b18f53f29d6126b/family1/124e1430343d49f4af3974cbd5e66b75, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithWeeklyPolicy/0758c3f3bb1fb9883b18f53f29d6126b/family1/c0203ba0be3a47bb9e3e061a0daa17e0, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithWeeklyPolicy/0758c3f3bb1fb9883b18f53f29d6126b/family1/3eca297c45f2403fa8dec972df370782, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithWeeklyPolicy/0758c3f3bb1fb9883b18f53f29d6126b/family1/af2fd9dd748e43e48dbbf3bbce516769, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithWeeklyPolicy/0758c3f3bb1fb9883b18f53f29d6126b/family1/7e93f5bb599c43598dcba71999291981, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithWeeklyPolicy/0758c3f3bb1fb9883b18f53f29d6126b/family1/e6cd24e717294644b7b7edafac286b07, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithWeeklyPolicy/0758c3f3bb1fb9883b18f53f29d6126b/family1/52ae76ef2c9747f8a8284b172db8f528, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithWeeklyPolicy/0758c3f3bb1fb9883b18f53f29d6126b/family1/ad3a32d354074bc6bf6bab1d686a143d_SeqId_44_] into tmpdir=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithWeeklyPolicy/0758c3f3bb1fb9883b18f53f29d6126b/.tmp, totalSize=50.0 K 2023-11-28 03:54:00,602 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] compactions.Compactor(207): Compacting 9683ab0ae7784e849f4f409399289833, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-11-28 03:54:00,602 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] compactions.Compactor(207): Compacting 1a92fa33465d4ad2bb8e9997f627f073, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=13 2023-11-28 03:54:00,603 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] compactions.Compactor(207): Compacting 124e1430343d49f4af3974cbd5e66b75, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=17 2023-11-28 03:54:00,603 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] compactions.Compactor(207): Compacting c0203ba0be3a47bb9e3e061a0daa17e0, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=21 2023-11-28 03:54:00,603 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] compactions.Compactor(207): Compacting 3eca297c45f2403fa8dec972df370782, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=25 2023-11-28 03:54:00,603 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] compactions.Compactor(207): Compacting af2fd9dd748e43e48dbbf3bbce516769, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=29 2023-11-28 03:54:00,603 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] compactions.Compactor(207): Compacting 7e93f5bb599c43598dcba71999291981, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=33 2023-11-28 03:54:00,603 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] compactions.Compactor(207): Compacting e6cd24e717294644b7b7edafac286b07, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=37 2023-11-28 03:54:00,603 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] compactions.Compactor(207): Compacting 52ae76ef2c9747f8a8284b172db8f528, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=41 2023-11-28 03:54:00,603 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] compactions.Compactor(207): Compacting ad3a32d354074bc6bf6bab1d686a143d_SeqId_44_, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=45 2023-11-28 03:54:00,604 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-11-28 03:54:00,605 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-28 03:54:00,607 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e2015122826ed36cdc1ad40a9839d9497ecd27a58 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e2015122826ed36cdc1ad40a9839d9497ecd27a58 2023-11-28 03:54:00,608 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e2015123183c771a5ce904cc48aad36bde7968168 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e2015123183c771a5ce904cc48aad36bde7968168 2023-11-28 03:54:00,610 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e2016010164d1f6ef7f0d40b7a753e00386bb8a0f to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e2016010164d1f6ef7f0d40b7a753e00386bb8a0f 2023-11-28 03:54:00,611 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20160103e83e5105757043849b95ce49688bb079 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20160103e83e5105757043849b95ce49688bb079 2023-11-28 03:54:00,611 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMinorCompactionWithWeeklyPolicy 4 expired mob files are deleted 2023-11-28 03:54:00,611 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228' 2023-11-28 03:54:00,612 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-11-28 03:54:00,612 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-11-28 03:54:00,612 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-11-28 03:54:00,612 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-11-28 03:54:00,612 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-11-28 03:54:00,612 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-11-28 03:54:00,612 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-11-28 03:54:00,612 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-11-28 03:54:00,645 INFO [RS:0;jenkins-hbase4:41203-shortCompactions-0] throttle.PressureAwareThroughputController(145): 0758c3f3bb1fb9883b18f53f29d6126b#family1#compaction#34 average throughput is 0.14 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-11-28 03:54:00,660 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205bde4b177e3574932abd399ddf733b4f2 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151205bde4b177e3574932abd399ddf733b4f2 2023-11-28 03:54:00,661 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-11-28 03:54:00,690 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/bc187c689b0c49b49fcf6eb2ee5c2699 first=Optional[r1] last=Optional[r2] 2023-11-28 03:54:00,691 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMinorCompactionWithWeeklyPolicy,,1701143628658.0758c3f3bb1fb9883b18f53f29d6126b., hostname=jenkins-hbase4.apache.org,41203,1701143600653, seqNum=2 for row with hfile group [{family1,hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/bc187c689b0c49b49fcf6eb2ee5c2699}] 2023-11-28 03:54:00,694 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HStore(763): Validating hfile at hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/bc187c689b0c49b49fcf6eb2ee5c2699 for inclusion in 0758c3f3bb1fb9883b18f53f29d6126b/family1 2023-11-28 03:54:00,697 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HStore(775): HFile bounds: first=r1 last=r2 2023-11-28 03:54:00,698 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HStore(777): Region bounds: first= last= 2023-11-28 03:54:00,698 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HRegion(2446): Flush status journal for 0758c3f3bb1fb9883b18f53f29d6126b: 2023-11-28 03:54:00,698 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/bc187c689b0c49b49fcf6eb2ee5c2699 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testMinorCompactionWithWeeklyPolicy__v7rbu5fhtqrtpdkvug8b84647v0hbmec6eldgo1sd0d83ebbu9sbtsf8f868mtd0/family1/bc187c689b0c49b49fcf6eb2ee5c2699 2023-11-28 03:54:00,701 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testMinorCompactionWithWeeklyPolicy__v7rbu5fhtqrtpdkvug8b84647v0hbmec6eldgo1sd0d83ebbu9sbtsf8f868mtd0/family1/bc187c689b0c49b49fcf6eb2ee5c2699 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithWeeklyPolicy/0758c3f3bb1fb9883b18f53f29d6126b/family1/72baf5c851e0461c84850143492db0be_SeqId_46_ 2023-11-28 03:54:00,701 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HStore(849): Loaded HFile hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testMinorCompactionWithWeeklyPolicy__v7rbu5fhtqrtpdkvug8b84647v0hbmec6eldgo1sd0d83ebbu9sbtsf8f868mtd0/family1/bc187c689b0c49b49fcf6eb2ee5c2699 into 0758c3f3bb1fb9883b18f53f29d6126b/family1 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithWeeklyPolicy/0758c3f3bb1fb9883b18f53f29d6126b/family1/72baf5c851e0461c84850143492db0be_SeqId_46_ - updating store file list. 2023-11-28 03:54:00,706 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HStore(882): Loaded HFile hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithWeeklyPolicy/0758c3f3bb1fb9883b18f53f29d6126b/family1/72baf5c851e0461c84850143492db0be_SeqId_46_ into 0758c3f3bb1fb9883b18f53f29d6126b/family1 2023-11-28 03:54:00,706 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HStore(855): Successfully loaded hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testMinorCompactionWithWeeklyPolicy__v7rbu5fhtqrtpdkvug8b84647v0hbmec6eldgo1sd0d83ebbu9sbtsf8f868mtd0/family1/bc187c689b0c49b49fcf6eb2ee5c2699 into 0758c3f3bb1fb9883b18f53f29d6126b/family1 (new location: hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithWeeklyPolicy/0758c3f3bb1fb9883b18f53f29d6126b/family1/72baf5c851e0461c84850143492db0be_SeqId_46_) 2023-11-28 03:54:00,707 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testMinorCompactionWithWeeklyPolicy__v7rbu5fhtqrtpdkvug8b84647v0hbmec6eldgo1sd0d83ebbu9sbtsf8f868mtd0/family1/bc187c689b0c49b49fcf6eb2ee5c2699 2023-11-28 03:54:00,709 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-11-28 03:54:00,709 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-28 03:54:00,711 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151201d1de3486eda14e3481c48434f1f6149a to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151201d1de3486eda14e3481c48434f1f6149a 2023-11-28 03:54:00,712 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e2015120537144d0f53624873972622c5e11ca323 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e2015120537144d0f53624873972622c5e11ca323 2023-11-28 03:54:00,712 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMinorCompactionWithWeeklyPolicy 2 expired mob files are deleted 2023-11-28 03:54:00,712 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130' 2023-11-28 03:54:00,713 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-11-28 03:54:00,713 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] compactions.PartitionedMobCompactor(354): After compaction, there are 5 mob files. table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-11-28 03:54:00,713 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] mob.MobUtils(840): end MOB compaction of files for table='testMinorCompactionWithWeeklyPolicy', column='family1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-11-28 03:54:00,714 DEBUG [PEWorker-2] locking.LockProcedure(242): UNLOCKED pid=26, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-11-28 03:54:00,716 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=26, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE in 1.0500 sec 2023-11-28 03:54:00,829 INFO [Listener at localhost/42853] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMinorCompactionWithWeeklyPolicy Thread=261 (was 260) - Thread LEAK? -, OpenFileDescriptor=651 (was 644) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=224 (was 259), ProcessCount=172 (was 172), AvailableMemoryMB=7028 (was 7128) 2023-11-28 03:54:00,842 INFO [Listener at localhost/42853] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testScannerOnBulkLoadRefHFiles Thread=261, OpenFileDescriptor=651, MaxFileDescriptor=60000, SystemLoadAverage=224, ProcessCount=172, AvailableMemoryMB=7027 2023-11-28 03:54:00,845 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testScannerOnBulkLoadRefHFiles', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} 2023-11-28 03:54:00,846 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] procedure2.ProcedureExecutor(1028): Stored pid=27, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles 2023-11-28 03:54:00,848 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_PRE_OPERATION 2023-11-28 03:54:00,848 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(700): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testScannerOnBulkLoadRefHFiles" procId is: 27 2023-11-28 03:54:00,849 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-11-28 03:54:00,849 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(1230): Checking to see if procedure is done pid=27 2023-11-28 03:54:00,852 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/.tmp/data/default/testScannerOnBulkLoadRefHFiles/a2457e9f3652153677ea1cb1286a9784 2023-11-28 03:54:00,852 DEBUG [HFileArchiver-6] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/.tmp/data/default/testScannerOnBulkLoadRefHFiles/303aa5c56b3058a5b63a20fa4b389c99 2023-11-28 03:54:00,852 DEBUG [HFileArchiver-7] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/.tmp/data/default/testScannerOnBulkLoadRefHFiles/02be414fee3293f77182043b018e4d6b 2023-11-28 03:54:00,852 DEBUG [HFileArchiver-5] backup.HFileArchiver(153): Directory hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/.tmp/data/default/testScannerOnBulkLoadRefHFiles/a2457e9f3652153677ea1cb1286a9784 empty. 2023-11-28 03:54:00,852 DEBUG [HFileArchiver-7] backup.HFileArchiver(153): Directory hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/.tmp/data/default/testScannerOnBulkLoadRefHFiles/02be414fee3293f77182043b018e4d6b empty. 2023-11-28 03:54:00,852 DEBUG [HFileArchiver-6] backup.HFileArchiver(153): Directory hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/.tmp/data/default/testScannerOnBulkLoadRefHFiles/303aa5c56b3058a5b63a20fa4b389c99 empty. 2023-11-28 03:54:00,853 DEBUG [HFileArchiver-5] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/.tmp/data/default/testScannerOnBulkLoadRefHFiles/a2457e9f3652153677ea1cb1286a9784 2023-11-28 03:54:00,853 DEBUG [HFileArchiver-7] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/.tmp/data/default/testScannerOnBulkLoadRefHFiles/02be414fee3293f77182043b018e4d6b 2023-11-28 03:54:00,853 DEBUG [HFileArchiver-6] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/.tmp/data/default/testScannerOnBulkLoadRefHFiles/303aa5c56b3058a5b63a20fa4b389c99 2023-11-28 03:54:00,853 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived testScannerOnBulkLoadRefHFiles regions 2023-11-28 03:54:00,950 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(1230): Checking to see if procedure is done pid=27 2023-11-28 03:54:01,065 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithWeeklyPolicy/0758c3f3bb1fb9883b18f53f29d6126b/.tmp/family1/2ac859b6cfde4f889e8bb5cac84edbf6 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithWeeklyPolicy/0758c3f3bb1fb9883b18f53f29d6126b/family1/2ac859b6cfde4f889e8bb5cac84edbf6 2023-11-28 03:54:01,072 INFO [RS:0;jenkins-hbase4:41203-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in 0758c3f3bb1fb9883b18f53f29d6126b/family1 of 0758c3f3bb1fb9883b18f53f29d6126b into 2ac859b6cfde4f889e8bb5cac84edbf6(size=6.6 K), total size for store is 176.8 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-11-28 03:54:01,072 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 0758c3f3bb1fb9883b18f53f29d6126b: 2023-11-28 03:54:01,072 INFO [RS:0;jenkins-hbase4:41203-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMinorCompactionWithWeeklyPolicy,,1701143628658.0758c3f3bb1fb9883b18f53f29d6126b., storeName=family1, fileCount=10, fileSize=50.0 K (5.0 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 5.5 K), priority=2, time=1701143640602; duration=0sec 2023-11-28 03:54:01,072 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-28 03:54:01,084 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithWeeklyPolicy/0758c3f3bb1fb9883b18f53f29d6126b/family1/9683ab0ae7784e849f4f409399289833, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithWeeklyPolicy/0758c3f3bb1fb9883b18f53f29d6126b/family1/1a92fa33465d4ad2bb8e9997f627f073, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithWeeklyPolicy/0758c3f3bb1fb9883b18f53f29d6126b/family1/124e1430343d49f4af3974cbd5e66b75, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithWeeklyPolicy/0758c3f3bb1fb9883b18f53f29d6126b/family1/c0203ba0be3a47bb9e3e061a0daa17e0, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithWeeklyPolicy/0758c3f3bb1fb9883b18f53f29d6126b/family1/3eca297c45f2403fa8dec972df370782, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithWeeklyPolicy/0758c3f3bb1fb9883b18f53f29d6126b/family1/af2fd9dd748e43e48dbbf3bbce516769, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithWeeklyPolicy/0758c3f3bb1fb9883b18f53f29d6126b/family1/7e93f5bb599c43598dcba71999291981, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithWeeklyPolicy/0758c3f3bb1fb9883b18f53f29d6126b/family1/e6cd24e717294644b7b7edafac286b07, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithWeeklyPolicy/0758c3f3bb1fb9883b18f53f29d6126b/family1/52ae76ef2c9747f8a8284b172db8f528, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithWeeklyPolicy/0758c3f3bb1fb9883b18f53f29d6126b/family1/ad3a32d354074bc6bf6bab1d686a143d_SeqId_44_] to archive 2023-11-28 03:54:01,085 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-28 03:54:01,087 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithWeeklyPolicy/0758c3f3bb1fb9883b18f53f29d6126b/family1/9683ab0ae7784e849f4f409399289833 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMinorCompactionWithWeeklyPolicy/0758c3f3bb1fb9883b18f53f29d6126b/family1/9683ab0ae7784e849f4f409399289833 2023-11-28 03:54:01,089 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithWeeklyPolicy/0758c3f3bb1fb9883b18f53f29d6126b/family1/1a92fa33465d4ad2bb8e9997f627f073 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMinorCompactionWithWeeklyPolicy/0758c3f3bb1fb9883b18f53f29d6126b/family1/1a92fa33465d4ad2bb8e9997f627f073 2023-11-28 03:54:01,091 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithWeeklyPolicy/0758c3f3bb1fb9883b18f53f29d6126b/family1/124e1430343d49f4af3974cbd5e66b75 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMinorCompactionWithWeeklyPolicy/0758c3f3bb1fb9883b18f53f29d6126b/family1/124e1430343d49f4af3974cbd5e66b75 2023-11-28 03:54:01,092 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithWeeklyPolicy/0758c3f3bb1fb9883b18f53f29d6126b/family1/c0203ba0be3a47bb9e3e061a0daa17e0 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMinorCompactionWithWeeklyPolicy/0758c3f3bb1fb9883b18f53f29d6126b/family1/c0203ba0be3a47bb9e3e061a0daa17e0 2023-11-28 03:54:01,094 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithWeeklyPolicy/0758c3f3bb1fb9883b18f53f29d6126b/family1/3eca297c45f2403fa8dec972df370782 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMinorCompactionWithWeeklyPolicy/0758c3f3bb1fb9883b18f53f29d6126b/family1/3eca297c45f2403fa8dec972df370782 2023-11-28 03:54:01,095 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithWeeklyPolicy/0758c3f3bb1fb9883b18f53f29d6126b/family1/af2fd9dd748e43e48dbbf3bbce516769 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMinorCompactionWithWeeklyPolicy/0758c3f3bb1fb9883b18f53f29d6126b/family1/af2fd9dd748e43e48dbbf3bbce516769 2023-11-28 03:54:01,097 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithWeeklyPolicy/0758c3f3bb1fb9883b18f53f29d6126b/family1/7e93f5bb599c43598dcba71999291981 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMinorCompactionWithWeeklyPolicy/0758c3f3bb1fb9883b18f53f29d6126b/family1/7e93f5bb599c43598dcba71999291981 2023-11-28 03:54:01,098 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithWeeklyPolicy/0758c3f3bb1fb9883b18f53f29d6126b/family1/e6cd24e717294644b7b7edafac286b07 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMinorCompactionWithWeeklyPolicy/0758c3f3bb1fb9883b18f53f29d6126b/family1/e6cd24e717294644b7b7edafac286b07 2023-11-28 03:54:01,099 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithWeeklyPolicy/0758c3f3bb1fb9883b18f53f29d6126b/family1/52ae76ef2c9747f8a8284b172db8f528 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMinorCompactionWithWeeklyPolicy/0758c3f3bb1fb9883b18f53f29d6126b/family1/52ae76ef2c9747f8a8284b172db8f528 2023-11-28 03:54:01,101 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithWeeklyPolicy/0758c3f3bb1fb9883b18f53f29d6126b/family1/ad3a32d354074bc6bf6bab1d686a143d_SeqId_44_ to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMinorCompactionWithWeeklyPolicy/0758c3f3bb1fb9883b18f53f29d6126b/family1/ad3a32d354074bc6bf6bab1d686a143d_SeqId_44_ 2023-11-28 03:54:01,151 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(1230): Checking to see if procedure is done pid=27 2023-11-28 03:54:01,262 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/.tmp/data/default/testScannerOnBulkLoadRefHFiles/.tabledesc/.tableinfo.0000000001 2023-11-28 03:54:01,264 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(7675): creating {ENCODED => 02be414fee3293f77182043b018e4d6b, NAME => 'testScannerOnBulkLoadRefHFiles,2,1701143640845.02be414fee3293f77182043b018e4d6b.', STARTKEY => '2', ENDKEY => ''}, tableDescriptor='testScannerOnBulkLoadRefHFiles', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/.tmp 2023-11-28 03:54:01,264 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(7675): creating {ENCODED => 303aa5c56b3058a5b63a20fa4b389c99, NAME => 'testScannerOnBulkLoadRefHFiles,1,1701143640845.303aa5c56b3058a5b63a20fa4b389c99.', STARTKEY => '1', ENDKEY => '2'}, tableDescriptor='testScannerOnBulkLoadRefHFiles', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/.tmp 2023-11-28 03:54:01,264 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(7675): creating {ENCODED => a2457e9f3652153677ea1cb1286a9784, NAME => 'testScannerOnBulkLoadRefHFiles,,1701143640845.a2457e9f3652153677ea1cb1286a9784.', STARTKEY => '', ENDKEY => '1'}, tableDescriptor='testScannerOnBulkLoadRefHFiles', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/.tmp 2023-11-28 03:54:01,293 INFO [Finalizer] client.ConnectionImplementation(1973): Closing master protocol: MasterService 2023-11-28 03:54:01,294 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x47660d3d to 127.0.0.1:55641 2023-11-28 03:54:01,294 DEBUG [Finalizer] ipc.AbstractRpcClient(489): Stopping rpc client 2023-11-28 03:54:01,295 INFO [Finalizer] client.ConnectionImplementation(1973): Closing master protocol: MasterService 2023-11-28 03:54:01,300 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x0329bf50 to 127.0.0.1:55641 2023-11-28 03:54:01,300 DEBUG [Finalizer] ipc.AbstractRpcClient(489): Stopping rpc client 2023-11-28 03:54:01,300 INFO [Finalizer] client.ConnectionImplementation(1973): Closing master protocol: MasterService 2023-11-28 03:54:01,300 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x56d83728 to 127.0.0.1:55641 2023-11-28 03:54:01,300 DEBUG [Finalizer] ipc.AbstractRpcClient(489): Stopping rpc client 2023-11-28 03:54:01,301 INFO [Finalizer] client.ConnectionImplementation(1973): Closing master protocol: MasterService 2023-11-28 03:54:01,301 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x51ad0549 to 127.0.0.1:55641 2023-11-28 03:54:01,302 DEBUG [Finalizer] ipc.AbstractRpcClient(489): Stopping rpc client 2023-11-28 03:54:01,302 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(866): Instantiated testScannerOnBulkLoadRefHFiles,1,1701143640845.303aa5c56b3058a5b63a20fa4b389c99.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-28 03:54:01,302 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1604): Closing 303aa5c56b3058a5b63a20fa4b389c99, disabling compactions & flushes 2023-11-28 03:54:01,302 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1626): Closing region testScannerOnBulkLoadRefHFiles,1,1701143640845.303aa5c56b3058a5b63a20fa4b389c99. 2023-11-28 03:54:01,302 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1647): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,1,1701143640845.303aa5c56b3058a5b63a20fa4b389c99. 2023-11-28 03:54:01,302 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1714): Acquired close lock on testScannerOnBulkLoadRefHFiles,1,1701143640845.303aa5c56b3058a5b63a20fa4b389c99. after waiting 0 ms 2023-11-28 03:54:01,302 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1724): Updates disabled for region testScannerOnBulkLoadRefHFiles,1,1701143640845.303aa5c56b3058a5b63a20fa4b389c99. 2023-11-28 03:54:01,303 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1838): Closed testScannerOnBulkLoadRefHFiles,1,1701143640845.303aa5c56b3058a5b63a20fa4b389c99. 2023-11-28 03:54:01,303 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1558): Region close journal for 303aa5c56b3058a5b63a20fa4b389c99: 2023-11-28 03:54:01,305 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(866): Instantiated testScannerOnBulkLoadRefHFiles,,1701143640845.a2457e9f3652153677ea1cb1286a9784.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-28 03:54:01,305 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1604): Closing a2457e9f3652153677ea1cb1286a9784, disabling compactions & flushes 2023-11-28 03:54:01,305 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1626): Closing region testScannerOnBulkLoadRefHFiles,,1701143640845.a2457e9f3652153677ea1cb1286a9784. 2023-11-28 03:54:01,305 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,,1701143640845.a2457e9f3652153677ea1cb1286a9784. 2023-11-28 03:54:01,305 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1714): Acquired close lock on testScannerOnBulkLoadRefHFiles,,1701143640845.a2457e9f3652153677ea1cb1286a9784. after waiting 0 ms 2023-11-28 03:54:01,305 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1724): Updates disabled for region testScannerOnBulkLoadRefHFiles,,1701143640845.a2457e9f3652153677ea1cb1286a9784. 2023-11-28 03:54:01,305 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1838): Closed testScannerOnBulkLoadRefHFiles,,1701143640845.a2457e9f3652153677ea1cb1286a9784. 2023-11-28 03:54:01,305 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1558): Region close journal for a2457e9f3652153677ea1cb1286a9784: 2023-11-28 03:54:01,452 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(1230): Checking to see if procedure is done pid=27 2023-11-28 03:54:01,700 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(866): Instantiated testScannerOnBulkLoadRefHFiles,2,1701143640845.02be414fee3293f77182043b018e4d6b.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-28 03:54:01,700 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1604): Closing 02be414fee3293f77182043b018e4d6b, disabling compactions & flushes 2023-11-28 03:54:01,700 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1626): Closing region testScannerOnBulkLoadRefHFiles,2,1701143640845.02be414fee3293f77182043b018e4d6b. 2023-11-28 03:54:01,700 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1647): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,2,1701143640845.02be414fee3293f77182043b018e4d6b. 2023-11-28 03:54:01,700 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1714): Acquired close lock on testScannerOnBulkLoadRefHFiles,2,1701143640845.02be414fee3293f77182043b018e4d6b. after waiting 0 ms 2023-11-28 03:54:01,700 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1724): Updates disabled for region testScannerOnBulkLoadRefHFiles,2,1701143640845.02be414fee3293f77182043b018e4d6b. 2023-11-28 03:54:01,700 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1838): Closed testScannerOnBulkLoadRefHFiles,2,1701143640845.02be414fee3293f77182043b018e4d6b. 2023-11-28 03:54:01,700 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1558): Region close journal for 02be414fee3293f77182043b018e4d6b: 2023-11-28 03:54:01,703 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_ADD_TO_META 2023-11-28 03:54:01,704 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testScannerOnBulkLoadRefHFiles,1,1701143640845.303aa5c56b3058a5b63a20fa4b389c99.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1701143641704"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1701143641704"}]},"ts":"1701143641704"} 2023-11-28 03:54:01,704 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testScannerOnBulkLoadRefHFiles,,1701143640845.a2457e9f3652153677ea1cb1286a9784.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1701143641704"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1701143641704"}]},"ts":"1701143641704"} 2023-11-28 03:54:01,704 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testScannerOnBulkLoadRefHFiles,2,1701143640845.02be414fee3293f77182043b018e4d6b.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1701143641704"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1701143641704"}]},"ts":"1701143641704"} 2023-11-28 03:54:01,707 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 3 regions to meta. 2023-11-28 03:54:01,708 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-11-28 03:54:01,708 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testScannerOnBulkLoadRefHFiles","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1701143641708"}]},"ts":"1701143641708"} 2023-11-28 03:54:01,709 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testScannerOnBulkLoadRefHFiles, state=ENABLING in hbase:meta 2023-11-28 03:54:01,719 INFO [PEWorker-3] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=28, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=a2457e9f3652153677ea1cb1286a9784, ASSIGN}, {pid=29, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=303aa5c56b3058a5b63a20fa4b389c99, ASSIGN}, {pid=30, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=02be414fee3293f77182043b018e4d6b, ASSIGN}] 2023-11-28 03:54:01,721 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=a2457e9f3652153677ea1cb1286a9784, ASSIGN 2023-11-28 03:54:01,721 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=303aa5c56b3058a5b63a20fa4b389c99, ASSIGN 2023-11-28 03:54:01,721 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=30, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=02be414fee3293f77182043b018e4d6b, ASSIGN 2023-11-28 03:54:01,722 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=29, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=303aa5c56b3058a5b63a20fa4b389c99, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,41203,1701143600653; forceNewPlan=false, retain=false 2023-11-28 03:54:01,722 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=30, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=02be414fee3293f77182043b018e4d6b, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,41203,1701143600653; forceNewPlan=false, retain=false 2023-11-28 03:54:01,722 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=28, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=a2457e9f3652153677ea1cb1286a9784, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,41203,1701143600653; forceNewPlan=false, retain=false 2023-11-28 03:54:01,873 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=28 updating hbase:meta row=a2457e9f3652153677ea1cb1286a9784, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:01,873 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=30 updating hbase:meta row=02be414fee3293f77182043b018e4d6b, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:01,873 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=303aa5c56b3058a5b63a20fa4b389c99, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:01,873 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testScannerOnBulkLoadRefHFiles,,1701143640845.a2457e9f3652153677ea1cb1286a9784.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1701143641873"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1701143641873"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1701143641873"}]},"ts":"1701143641873"} 2023-11-28 03:54:01,874 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testScannerOnBulkLoadRefHFiles,2,1701143640845.02be414fee3293f77182043b018e4d6b.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1701143641873"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1701143641873"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1701143641873"}]},"ts":"1701143641873"} 2023-11-28 03:54:01,874 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testScannerOnBulkLoadRefHFiles,1,1701143640845.303aa5c56b3058a5b63a20fa4b389c99.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1701143641873"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1701143641873"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1701143641873"}]},"ts":"1701143641873"} 2023-11-28 03:54:01,875 INFO [PEWorker-2] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=31, ppid=28, state=RUNNABLE; OpenRegionProcedure a2457e9f3652153677ea1cb1286a9784, server=jenkins-hbase4.apache.org,41203,1701143600653}] 2023-11-28 03:54:01,876 INFO [PEWorker-3] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=32, ppid=30, state=RUNNABLE; OpenRegionProcedure 02be414fee3293f77182043b018e4d6b, server=jenkins-hbase4.apache.org,41203,1701143600653}] 2023-11-28 03:54:01,878 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=33, ppid=29, state=RUNNABLE; OpenRegionProcedure 303aa5c56b3058a5b63a20fa4b389c99, server=jenkins-hbase4.apache.org,41203,1701143600653}] 2023-11-28 03:54:01,953 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(1230): Checking to see if procedure is done pid=27 2023-11-28 03:54:02,027 DEBUG [RSProcedureDispatcher-pool-1] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:02,030 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testScannerOnBulkLoadRefHFiles,2,1701143640845.02be414fee3293f77182043b018e4d6b. 2023-11-28 03:54:02,031 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 02be414fee3293f77182043b018e4d6b, NAME => 'testScannerOnBulkLoadRefHFiles,2,1701143640845.02be414fee3293f77182043b018e4d6b.', STARTKEY => '2', ENDKEY => ''} 2023-11-28 03:54:02,031 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testScannerOnBulkLoadRefHFiles 02be414fee3293f77182043b018e4d6b 2023-11-28 03:54:02,031 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testScannerOnBulkLoadRefHFiles,2,1701143640845.02be414fee3293f77182043b018e4d6b.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-28 03:54:02,031 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 02be414fee3293f77182043b018e4d6b 2023-11-28 03:54:02,031 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 02be414fee3293f77182043b018e4d6b 2023-11-28 03:54:02,032 INFO [StoreOpener-02be414fee3293f77182043b018e4d6b-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 02be414fee3293f77182043b018e4d6b 2023-11-28 03:54:02,034 DEBUG [StoreOpener-02be414fee3293f77182043b018e4d6b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testScannerOnBulkLoadRefHFiles/02be414fee3293f77182043b018e4d6b/family1 2023-11-28 03:54:02,034 DEBUG [StoreOpener-02be414fee3293f77182043b018e4d6b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testScannerOnBulkLoadRefHFiles/02be414fee3293f77182043b018e4d6b/family1 2023-11-28 03:54:02,034 INFO [StoreOpener-02be414fee3293f77182043b018e4d6b-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 02be414fee3293f77182043b018e4d6b columnFamilyName family1 2023-11-28 03:54:02,036 INFO [StoreOpener-02be414fee3293f77182043b018e4d6b-1] regionserver.HStore(310): Store=02be414fee3293f77182043b018e4d6b/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-28 03:54:02,036 INFO [StoreOpener-02be414fee3293f77182043b018e4d6b-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 02be414fee3293f77182043b018e4d6b 2023-11-28 03:54:02,037 DEBUG [StoreOpener-02be414fee3293f77182043b018e4d6b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testScannerOnBulkLoadRefHFiles/02be414fee3293f77182043b018e4d6b/family2 2023-11-28 03:54:02,037 DEBUG [StoreOpener-02be414fee3293f77182043b018e4d6b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testScannerOnBulkLoadRefHFiles/02be414fee3293f77182043b018e4d6b/family2 2023-11-28 03:54:02,038 INFO [StoreOpener-02be414fee3293f77182043b018e4d6b-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 02be414fee3293f77182043b018e4d6b columnFamilyName family2 2023-11-28 03:54:02,039 INFO [StoreOpener-02be414fee3293f77182043b018e4d6b-1] regionserver.HStore(310): Store=02be414fee3293f77182043b018e4d6b/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-28 03:54:02,040 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testScannerOnBulkLoadRefHFiles/02be414fee3293f77182043b018e4d6b 2023-11-28 03:54:02,040 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testScannerOnBulkLoadRefHFiles/02be414fee3293f77182043b018e4d6b 2023-11-28 03:54:02,042 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testScannerOnBulkLoadRefHFiles descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-11-28 03:54:02,043 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 02be414fee3293f77182043b018e4d6b 2023-11-28 03:54:02,045 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testScannerOnBulkLoadRefHFiles/02be414fee3293f77182043b018e4d6b/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-11-28 03:54:02,045 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 02be414fee3293f77182043b018e4d6b; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=70363738, jitterRate=0.048501402139663696}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-11-28 03:54:02,045 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 02be414fee3293f77182043b018e4d6b: 2023-11-28 03:54:02,046 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2339): Post open deploy tasks for testScannerOnBulkLoadRefHFiles,2,1701143640845.02be414fee3293f77182043b018e4d6b., pid=32, masterSystemTime=1701143642027 2023-11-28 03:54:02,048 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2366): Finished post open deploy task for testScannerOnBulkLoadRefHFiles,2,1701143640845.02be414fee3293f77182043b018e4d6b. 2023-11-28 03:54:02,048 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testScannerOnBulkLoadRefHFiles,2,1701143640845.02be414fee3293f77182043b018e4d6b. 2023-11-28 03:54:02,048 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testScannerOnBulkLoadRefHFiles,,1701143640845.a2457e9f3652153677ea1cb1286a9784. 2023-11-28 03:54:02,048 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => a2457e9f3652153677ea1cb1286a9784, NAME => 'testScannerOnBulkLoadRefHFiles,,1701143640845.a2457e9f3652153677ea1cb1286a9784.', STARTKEY => '', ENDKEY => '1'} 2023-11-28 03:54:02,048 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testScannerOnBulkLoadRefHFiles a2457e9f3652153677ea1cb1286a9784 2023-11-28 03:54:02,048 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testScannerOnBulkLoadRefHFiles,,1701143640845.a2457e9f3652153677ea1cb1286a9784.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-28 03:54:02,048 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=30 updating hbase:meta row=02be414fee3293f77182043b018e4d6b, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:02,048 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for a2457e9f3652153677ea1cb1286a9784 2023-11-28 03:54:02,049 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for a2457e9f3652153677ea1cb1286a9784 2023-11-28 03:54:02,049 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testScannerOnBulkLoadRefHFiles,2,1701143640845.02be414fee3293f77182043b018e4d6b.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1701143642048"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1701143642048"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1701143642048"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1701143642048"}]},"ts":"1701143642048"} 2023-11-28 03:54:02,050 INFO [StoreOpener-a2457e9f3652153677ea1cb1286a9784-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region a2457e9f3652153677ea1cb1286a9784 2023-11-28 03:54:02,051 DEBUG [StoreOpener-a2457e9f3652153677ea1cb1286a9784-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testScannerOnBulkLoadRefHFiles/a2457e9f3652153677ea1cb1286a9784/family1 2023-11-28 03:54:02,051 DEBUG [StoreOpener-a2457e9f3652153677ea1cb1286a9784-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testScannerOnBulkLoadRefHFiles/a2457e9f3652153677ea1cb1286a9784/family1 2023-11-28 03:54:02,052 INFO [StoreOpener-a2457e9f3652153677ea1cb1286a9784-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a2457e9f3652153677ea1cb1286a9784 columnFamilyName family1 2023-11-28 03:54:02,052 INFO [PEWorker-1] procedure2.ProcedureExecutor(1825): Finished subprocedure pid=32, resume processing ppid=30 2023-11-28 03:54:02,052 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=32, ppid=30, state=SUCCESS; OpenRegionProcedure 02be414fee3293f77182043b018e4d6b, server=jenkins-hbase4.apache.org,41203,1701143600653 in 174 msec 2023-11-28 03:54:02,053 INFO [StoreOpener-a2457e9f3652153677ea1cb1286a9784-1] regionserver.HStore(310): Store=a2457e9f3652153677ea1cb1286a9784/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-28 03:54:02,053 INFO [StoreOpener-a2457e9f3652153677ea1cb1286a9784-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region a2457e9f3652153677ea1cb1286a9784 2023-11-28 03:54:02,053 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=30, ppid=27, state=SUCCESS; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=02be414fee3293f77182043b018e4d6b, ASSIGN in 333 msec 2023-11-28 03:54:02,055 DEBUG [StoreOpener-a2457e9f3652153677ea1cb1286a9784-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testScannerOnBulkLoadRefHFiles/a2457e9f3652153677ea1cb1286a9784/family2 2023-11-28 03:54:02,055 DEBUG [StoreOpener-a2457e9f3652153677ea1cb1286a9784-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testScannerOnBulkLoadRefHFiles/a2457e9f3652153677ea1cb1286a9784/family2 2023-11-28 03:54:02,055 INFO [StoreOpener-a2457e9f3652153677ea1cb1286a9784-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a2457e9f3652153677ea1cb1286a9784 columnFamilyName family2 2023-11-28 03:54:02,056 INFO [StoreOpener-a2457e9f3652153677ea1cb1286a9784-1] regionserver.HStore(310): Store=a2457e9f3652153677ea1cb1286a9784/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-28 03:54:02,057 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testScannerOnBulkLoadRefHFiles/a2457e9f3652153677ea1cb1286a9784 2023-11-28 03:54:02,057 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testScannerOnBulkLoadRefHFiles/a2457e9f3652153677ea1cb1286a9784 2023-11-28 03:54:02,059 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testScannerOnBulkLoadRefHFiles descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-11-28 03:54:02,060 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for a2457e9f3652153677ea1cb1286a9784 2023-11-28 03:54:02,061 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testScannerOnBulkLoadRefHFiles/a2457e9f3652153677ea1cb1286a9784/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-11-28 03:54:02,062 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened a2457e9f3652153677ea1cb1286a9784; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=73299755, jitterRate=0.09225146472454071}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-11-28 03:54:02,062 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for a2457e9f3652153677ea1cb1286a9784: 2023-11-28 03:54:02,062 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2339): Post open deploy tasks for testScannerOnBulkLoadRefHFiles,,1701143640845.a2457e9f3652153677ea1cb1286a9784., pid=31, masterSystemTime=1701143642027 2023-11-28 03:54:02,064 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2366): Finished post open deploy task for testScannerOnBulkLoadRefHFiles,,1701143640845.a2457e9f3652153677ea1cb1286a9784. 2023-11-28 03:54:02,064 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testScannerOnBulkLoadRefHFiles,,1701143640845.a2457e9f3652153677ea1cb1286a9784. 2023-11-28 03:54:02,064 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testScannerOnBulkLoadRefHFiles,1,1701143640845.303aa5c56b3058a5b63a20fa4b389c99. 2023-11-28 03:54:02,064 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 303aa5c56b3058a5b63a20fa4b389c99, NAME => 'testScannerOnBulkLoadRefHFiles,1,1701143640845.303aa5c56b3058a5b63a20fa4b389c99.', STARTKEY => '1', ENDKEY => '2'} 2023-11-28 03:54:02,064 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testScannerOnBulkLoadRefHFiles 303aa5c56b3058a5b63a20fa4b389c99 2023-11-28 03:54:02,064 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=28 updating hbase:meta row=a2457e9f3652153677ea1cb1286a9784, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:02,064 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testScannerOnBulkLoadRefHFiles,1,1701143640845.303aa5c56b3058a5b63a20fa4b389c99.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-28 03:54:02,064 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 303aa5c56b3058a5b63a20fa4b389c99 2023-11-28 03:54:02,064 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 303aa5c56b3058a5b63a20fa4b389c99 2023-11-28 03:54:02,064 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testScannerOnBulkLoadRefHFiles,,1701143640845.a2457e9f3652153677ea1cb1286a9784.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1701143642064"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1701143642064"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1701143642064"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1701143642064"}]},"ts":"1701143642064"} 2023-11-28 03:54:02,065 INFO [StoreOpener-303aa5c56b3058a5b63a20fa4b389c99-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 303aa5c56b3058a5b63a20fa4b389c99 2023-11-28 03:54:02,067 DEBUG [StoreOpener-303aa5c56b3058a5b63a20fa4b389c99-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testScannerOnBulkLoadRefHFiles/303aa5c56b3058a5b63a20fa4b389c99/family1 2023-11-28 03:54:02,067 DEBUG [StoreOpener-303aa5c56b3058a5b63a20fa4b389c99-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testScannerOnBulkLoadRefHFiles/303aa5c56b3058a5b63a20fa4b389c99/family1 2023-11-28 03:54:02,067 INFO [StoreOpener-303aa5c56b3058a5b63a20fa4b389c99-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 303aa5c56b3058a5b63a20fa4b389c99 columnFamilyName family1 2023-11-28 03:54:02,067 INFO [PEWorker-5] procedure2.ProcedureExecutor(1825): Finished subprocedure pid=31, resume processing ppid=28 2023-11-28 03:54:02,067 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=31, ppid=28, state=SUCCESS; OpenRegionProcedure a2457e9f3652153677ea1cb1286a9784, server=jenkins-hbase4.apache.org,41203,1701143600653 in 191 msec 2023-11-28 03:54:02,068 INFO [StoreOpener-303aa5c56b3058a5b63a20fa4b389c99-1] regionserver.HStore(310): Store=303aa5c56b3058a5b63a20fa4b389c99/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-28 03:54:02,068 INFO [StoreOpener-303aa5c56b3058a5b63a20fa4b389c99-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 303aa5c56b3058a5b63a20fa4b389c99 2023-11-28 03:54:02,069 INFO [PEWorker-4] procedure2.ProcedureExecutor(1409): Finished pid=28, ppid=27, state=SUCCESS; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=a2457e9f3652153677ea1cb1286a9784, ASSIGN in 348 msec 2023-11-28 03:54:02,070 DEBUG [StoreOpener-303aa5c56b3058a5b63a20fa4b389c99-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testScannerOnBulkLoadRefHFiles/303aa5c56b3058a5b63a20fa4b389c99/family2 2023-11-28 03:54:02,070 DEBUG [StoreOpener-303aa5c56b3058a5b63a20fa4b389c99-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testScannerOnBulkLoadRefHFiles/303aa5c56b3058a5b63a20fa4b389c99/family2 2023-11-28 03:54:02,070 INFO [StoreOpener-303aa5c56b3058a5b63a20fa4b389c99-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 303aa5c56b3058a5b63a20fa4b389c99 columnFamilyName family2 2023-11-28 03:54:02,071 INFO [StoreOpener-303aa5c56b3058a5b63a20fa4b389c99-1] regionserver.HStore(310): Store=303aa5c56b3058a5b63a20fa4b389c99/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-28 03:54:02,071 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testScannerOnBulkLoadRefHFiles/303aa5c56b3058a5b63a20fa4b389c99 2023-11-28 03:54:02,072 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testScannerOnBulkLoadRefHFiles/303aa5c56b3058a5b63a20fa4b389c99 2023-11-28 03:54:02,073 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testScannerOnBulkLoadRefHFiles descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-11-28 03:54:02,074 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 303aa5c56b3058a5b63a20fa4b389c99 2023-11-28 03:54:02,076 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testScannerOnBulkLoadRefHFiles/303aa5c56b3058a5b63a20fa4b389c99/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-11-28 03:54:02,076 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 303aa5c56b3058a5b63a20fa4b389c99; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=62982590, jitterRate=-0.061486274003982544}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-11-28 03:54:02,076 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 303aa5c56b3058a5b63a20fa4b389c99: 2023-11-28 03:54:02,077 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2339): Post open deploy tasks for testScannerOnBulkLoadRefHFiles,1,1701143640845.303aa5c56b3058a5b63a20fa4b389c99., pid=33, masterSystemTime=1701143642027 2023-11-28 03:54:02,078 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2366): Finished post open deploy task for testScannerOnBulkLoadRefHFiles,1,1701143640845.303aa5c56b3058a5b63a20fa4b389c99. 2023-11-28 03:54:02,078 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testScannerOnBulkLoadRefHFiles,1,1701143640845.303aa5c56b3058a5b63a20fa4b389c99. 2023-11-28 03:54:02,079 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=303aa5c56b3058a5b63a20fa4b389c99, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:02,079 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testScannerOnBulkLoadRefHFiles,1,1701143640845.303aa5c56b3058a5b63a20fa4b389c99.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1701143642078"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1701143642078"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1701143642078"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1701143642078"}]},"ts":"1701143642078"} 2023-11-28 03:54:02,081 INFO [PEWorker-3] procedure2.ProcedureExecutor(1825): Finished subprocedure pid=33, resume processing ppid=29 2023-11-28 03:54:02,082 INFO [PEWorker-3] procedure2.ProcedureExecutor(1409): Finished pid=33, ppid=29, state=SUCCESS; OpenRegionProcedure 303aa5c56b3058a5b63a20fa4b389c99, server=jenkins-hbase4.apache.org,41203,1701143600653 in 203 msec 2023-11-28 03:54:02,083 INFO [PEWorker-1] procedure2.ProcedureExecutor(1825): Finished subprocedure pid=29, resume processing ppid=27 2023-11-28 03:54:02,083 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=29, ppid=27, state=SUCCESS; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=303aa5c56b3058a5b63a20fa4b389c99, ASSIGN in 363 msec 2023-11-28 03:54:02,084 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-11-28 03:54:02,084 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testScannerOnBulkLoadRefHFiles","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1701143642084"}]},"ts":"1701143642084"} 2023-11-28 03:54:02,085 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testScannerOnBulkLoadRefHFiles, state=ENABLED in hbase:meta 2023-11-28 03:54:02,088 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_POST_OPERATION 2023-11-28 03:54:02,089 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=27, state=SUCCESS; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles in 1.2430 sec 2023-11-28 03:54:02,954 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(1230): Checking to see if procedure is done pid=27 2023-11-28 03:54:02,955 INFO [Listener at localhost/42853] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testScannerOnBulkLoadRefHFiles, procId: 27 completed 2023-11-28 03:54:02,962 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-28 03:54:02,964 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] procedure.ProcedureCoordinator(165): Submitting procedure testScannerOnBulkLoadRefHFiles 2023-11-28 03:54:02,964 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testScannerOnBulkLoadRefHFiles' 2023-11-28 03:54:02,964 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:54:02,964 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testScannerOnBulkLoadRefHFiles' starting 'acquire' 2023-11-28 03:54:02,964 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testScannerOnBulkLoadRefHFiles', kicking off acquire phase on members. 2023-11-28 03:54:02,965 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-11-28 03:54:02,965 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-11-28 03:54:02,966 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:02,966 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:54:02,967 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:54:02,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:54:02,967 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:02,967 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-28 03:54:02,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-11-28 03:54:02,967 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-11-28 03:54:02,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-28 03:54:02,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-11-28 03:54:02,968 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testScannerOnBulkLoadRefHFiles 2023-11-28 03:54:02,968 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testScannerOnBulkLoadRefHFiles 2023-11-28 03:54:02,968 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testScannerOnBulkLoadRefHFiles' with timeout 60000ms 2023-11-28 03:54:02,968 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:54:02,968 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testScannerOnBulkLoadRefHFiles' starting 'acquire' stage 2023-11-28 03:54:02,968 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,1,1701143640845.303aa5c56b3058a5b63a20fa4b389c99. 2023-11-28 03:54:02,968 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,,1701143640845.a2457e9f3652153677ea1cb1286a9784. 2023-11-28 03:54:02,968 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,,1701143640845.a2457e9f3652153677ea1cb1286a9784. started... 2023-11-28 03:54:02,968 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,1,1701143640845.303aa5c56b3058a5b63a20fa4b389c99. started... 2023-11-28 03:54:02,969 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,2,1701143640845.02be414fee3293f77182043b018e4d6b. 2023-11-28 03:54:02,969 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for a2457e9f3652153677ea1cb1286a9784: 2023-11-28 03:54:02,968 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-11-28 03:54:02,970 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,,1701143640845.a2457e9f3652153677ea1cb1286a9784. 2023-11-28 03:54:02,969 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 303aa5c56b3058a5b63a20fa4b389c99: 2023-11-28 03:54:02,969 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,2,1701143640845.02be414fee3293f77182043b018e4d6b. started... 2023-11-28 03:54:02,970 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,1,1701143640845.303aa5c56b3058a5b63a20fa4b389c99. 2023-11-28 03:54:02,970 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-28 03:54:02,970 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 02be414fee3293f77182043b018e4d6b 2/2 column families, dataSize=48 B heapSize=632 B 2023-11-28 03:54:02,970 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-11-28 03:54:02,970 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-11-28 03:54:02,985 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023112823a282d0bb204bcd87fd14585188d0e9 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c2023112823a282d0bb204bcd87fd14585188d0e9 2023-11-28 03:54:03,391 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=48, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testScannerOnBulkLoadRefHFiles/02be414fee3293f77182043b018e4d6b/.tmp/family1/1f7cad3095c64232b280fd227f7cb144 2023-11-28 03:54:03,396 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testScannerOnBulkLoadRefHFiles/02be414fee3293f77182043b018e4d6b/.tmp/family1/1f7cad3095c64232b280fd227f7cb144 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testScannerOnBulkLoadRefHFiles/02be414fee3293f77182043b018e4d6b/family1/1f7cad3095c64232b280fd227f7cb144 2023-11-28 03:54:03,401 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testScannerOnBulkLoadRefHFiles/02be414fee3293f77182043b018e4d6b/family1/1f7cad3095c64232b280fd227f7cb144, entries=1, sequenceid=5, filesize=4.9 K 2023-11-28 03:54:03,402 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~48 B/48, heapSize ~360 B/360, currentSize=0 B/0 for 02be414fee3293f77182043b018e4d6b in 432ms, sequenceid=5, compaction requested=false 2023-11-28 03:54:03,402 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testScannerOnBulkLoadRefHFiles' 2023-11-28 03:54:03,403 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 02be414fee3293f77182043b018e4d6b: 2023-11-28 03:54:03,403 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,2,1701143640845.02be414fee3293f77182043b018e4d6b. 2023-11-28 03:54:03,403 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-11-28 03:54:03,403 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-11-28 03:54:03,403 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:03,403 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testScannerOnBulkLoadRefHFiles' locally acquired 2023-11-28 03:54:03,403 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure (testScannerOnBulkLoadRefHFiles) in zk 2023-11-28 03:54:03,405 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:03,405 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-11-28 03:54:03,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:03,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:54:03,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:54:03,405 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-11-28 03:54:03,405 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testScannerOnBulkLoadRefHFiles' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-28 03:54:03,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:54:03,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:54:03,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-11-28 03:54:03,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:03,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:54:03,407 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure 'testScannerOnBulkLoadRefHFiles' on coordinator 2023-11-28 03:54:03,407 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@72198ef1[Count = 0] remaining members to acquire global barrier 2023-11-28 03:54:03,407 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testScannerOnBulkLoadRefHFiles' starting 'in-barrier' execution. 2023-11-28 03:54:03,407 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-11-28 03:54:03,409 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-11-28 03:54:03,409 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-11-28 03:54:03,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-11-28 03:54:03,409 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testScannerOnBulkLoadRefHFiles' received 'reached' from coordinator. 2023-11-28 03:54:03,409 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testScannerOnBulkLoadRefHFiles' locally completed 2023-11-28 03:54:03,409 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testScannerOnBulkLoadRefHFiles' completed for member 'jenkins-hbase4.apache.org,41203,1701143600653' in zk 2023-11-28 03:54:03,409 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:03,409 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-28 03:54:03,411 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:03,411 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testScannerOnBulkLoadRefHFiles' has notified controller of completion 2023-11-28 03:54:03,411 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:54:03,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:03,411 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testScannerOnBulkLoadRefHFiles' completed. 2023-11-28 03:54:03,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:54:03,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:54:03,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:54:03,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:54:03,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-11-28 03:54:03,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:03,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:54:03,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-11-28 03:54:03,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:03,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testScannerOnBulkLoadRefHFiles' member 'jenkins-hbase4.apache.org,41203,1701143600653': 2023-11-28 03:54:03,414 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' released barrier for procedure'testScannerOnBulkLoadRefHFiles', counting down latch. Waiting for 0 more 2023-11-28 03:54:03,414 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testScannerOnBulkLoadRefHFiles' execution completed 2023-11-28 03:54:03,414 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-28 03:54:03,414 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-28 03:54:03,414 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testScannerOnBulkLoadRefHFiles 2023-11-28 03:54:03,414 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testScannerOnBulkLoadRefHFilesincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-28 03:54:03,416 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-11-28 03:54:03,416 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-11-28 03:54:03,416 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-11-28 03:54:03,416 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:54:03,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-11-28 03:54:03,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-11-28 03:54:03,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:54:03,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:54:03,417 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:03,417 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:54:03,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:54:03,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:54:03,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-11-28 03:54:03,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-11-28 03:54:03,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:54:03,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-11-28 03:54:03,418 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:03,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:03,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:54:03,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-11-28 03:54:03,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:03,421 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:03,421 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:54:03,421 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-11-28 03:54:03,421 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testScannerOnBulkLoadRefHFiles' 2023-11-28 03:54:03,421 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-28 03:54:03,421 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-11-28 03:54:03,421 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(399): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles because node does not exist (not an error) 2023-11-28 03:54:03,421 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:54:03,421 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:54:03,422 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testScannerOnBulkLoadRefHFiles'' to complete. (max 300000 ms per retry) 2023-11-28 03:54:03,422 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-28 03:54:03,421 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:54:03,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:54:03,422 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:03,422 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-11-28 03:54:03,422 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-11-28 03:54:03,422 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-11-28 03:54:03,422 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:54:03,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:54:03,522 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-28 03:54:03,523 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-28 03:54:03,527 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-28 03:54:03,530 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] procedure.ProcedureCoordinator(143): Procedure testScannerOnBulkLoadRefHFiles was in running list but was completed. Accepting new attempt. 2023-11-28 03:54:03,530 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] procedure.ProcedureCoordinator(165): Submitting procedure testScannerOnBulkLoadRefHFiles 2023-11-28 03:54:03,530 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testScannerOnBulkLoadRefHFiles' 2023-11-28 03:54:03,530 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:54:03,530 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testScannerOnBulkLoadRefHFiles' starting 'acquire' 2023-11-28 03:54:03,530 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testScannerOnBulkLoadRefHFiles', kicking off acquire phase on members. 2023-11-28 03:54:03,530 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-11-28 03:54:03,530 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-11-28 03:54:03,533 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:03,533 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:54:03,533 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:54:03,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:54:03,533 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:03,533 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-28 03:54:03,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-11-28 03:54:03,534 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-11-28 03:54:03,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-28 03:54:03,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-11-28 03:54:03,534 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testScannerOnBulkLoadRefHFiles 2023-11-28 03:54:03,534 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testScannerOnBulkLoadRefHFiles is still present, removing 2023-11-28 03:54:03,534 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testScannerOnBulkLoadRefHFiles 2023-11-28 03:54:03,535 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testScannerOnBulkLoadRefHFiles' with timeout 60000ms 2023-11-28 03:54:03,535 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:54:03,535 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testScannerOnBulkLoadRefHFiles' starting 'acquire' stage 2023-11-28 03:54:03,535 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,1,1701143640845.303aa5c56b3058a5b63a20fa4b389c99. 2023-11-28 03:54:03,535 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,1,1701143640845.303aa5c56b3058a5b63a20fa4b389c99. started... 2023-11-28 03:54:03,535 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 303aa5c56b3058a5b63a20fa4b389c99: 2023-11-28 03:54:03,535 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,1,1701143640845.303aa5c56b3058a5b63a20fa4b389c99. 2023-11-28 03:54:03,536 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,,1701143640845.a2457e9f3652153677ea1cb1286a9784. 2023-11-28 03:54:03,536 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,,1701143640845.a2457e9f3652153677ea1cb1286a9784. started... 2023-11-28 03:54:03,536 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-11-28 03:54:03,536 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-28 03:54:03,536 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for a2457e9f3652153677ea1cb1286a9784: 2023-11-28 03:54:03,536 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,2,1701143640845.02be414fee3293f77182043b018e4d6b. 2023-11-28 03:54:03,536 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-11-28 03:54:03,537 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,2,1701143640845.02be414fee3293f77182043b018e4d6b. started... 2023-11-28 03:54:03,536 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,,1701143640845.a2457e9f3652153677ea1cb1286a9784. 2023-11-28 03:54:03,537 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 02be414fee3293f77182043b018e4d6b 2/2 column families, dataSize=90 B heapSize=752 B 2023-11-28 03:54:03,537 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-11-28 03:54:03,554 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023112899aa86221a204c73bd94f5f3fe7d5016 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c2023112899aa86221a204c73bd94f5f3fe7d5016 2023-11-28 03:54:03,960 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=90, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testScannerOnBulkLoadRefHFiles/02be414fee3293f77182043b018e4d6b/.tmp/family1/e2b38dc5c6894cda8fb11afc60de22f5 2023-11-28 03:54:03,965 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testScannerOnBulkLoadRefHFiles/02be414fee3293f77182043b018e4d6b/.tmp/family1/e2b38dc5c6894cda8fb11afc60de22f5 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testScannerOnBulkLoadRefHFiles/02be414fee3293f77182043b018e4d6b/family1/e2b38dc5c6894cda8fb11afc60de22f5 2023-11-28 03:54:03,970 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testScannerOnBulkLoadRefHFiles/02be414fee3293f77182043b018e4d6b/family1/e2b38dc5c6894cda8fb11afc60de22f5, entries=2, sequenceid=9, filesize=5.0 K 2023-11-28 03:54:03,971 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~90 B/90, heapSize ~480 B/480, currentSize=0 B/0 for 02be414fee3293f77182043b018e4d6b in 434ms, sequenceid=9, compaction requested=false 2023-11-28 03:54:03,972 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 02be414fee3293f77182043b018e4d6b: 2023-11-28 03:54:03,972 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,2,1701143640845.02be414fee3293f77182043b018e4d6b. 2023-11-28 03:54:03,972 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-11-28 03:54:03,972 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-11-28 03:54:03,972 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:03,972 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testScannerOnBulkLoadRefHFiles' locally acquired 2023-11-28 03:54:03,972 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure (testScannerOnBulkLoadRefHFiles) in zk 2023-11-28 03:54:03,974 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-11-28 03:54:03,974 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:03,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:03,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:54:03,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:54:03,974 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-11-28 03:54:03,974 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testScannerOnBulkLoadRefHFiles' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-28 03:54:03,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:54:03,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:54:03,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-11-28 03:54:03,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:03,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:54:03,976 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure 'testScannerOnBulkLoadRefHFiles' on coordinator 2023-11-28 03:54:03,976 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@24d0a653[Count = 0] remaining members to acquire global barrier 2023-11-28 03:54:03,976 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testScannerOnBulkLoadRefHFiles' starting 'in-barrier' execution. 2023-11-28 03:54:03,977 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-11-28 03:54:03,978 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-11-28 03:54:03,978 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-11-28 03:54:03,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-11-28 03:54:03,978 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testScannerOnBulkLoadRefHFiles' received 'reached' from coordinator. 2023-11-28 03:54:03,978 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testScannerOnBulkLoadRefHFiles' locally completed 2023-11-28 03:54:03,978 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testScannerOnBulkLoadRefHFiles' completed for member 'jenkins-hbase4.apache.org,41203,1701143600653' in zk 2023-11-28 03:54:03,978 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:03,978 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-28 03:54:03,980 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testScannerOnBulkLoadRefHFiles' has notified controller of completion 2023-11-28 03:54:03,980 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:03,980 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:54:03,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:03,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:54:03,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:54:03,980 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testScannerOnBulkLoadRefHFiles' completed. 2023-11-28 03:54:03,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:54:03,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:54:03,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-11-28 03:54:03,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:03,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:54:03,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-11-28 03:54:03,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:03,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testScannerOnBulkLoadRefHFiles' member 'jenkins-hbase4.apache.org,41203,1701143600653': 2023-11-28 03:54:03,983 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' released barrier for procedure'testScannerOnBulkLoadRefHFiles', counting down latch. Waiting for 0 more 2023-11-28 03:54:03,983 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testScannerOnBulkLoadRefHFiles' execution completed 2023-11-28 03:54:03,983 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-28 03:54:03,983 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-28 03:54:03,983 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testScannerOnBulkLoadRefHFiles 2023-11-28 03:54:03,983 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testScannerOnBulkLoadRefHFilesincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-28 03:54:03,989 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-11-28 03:54:03,989 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-11-28 03:54:03,989 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-11-28 03:54:03,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-11-28 03:54:03,989 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:54:03,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-11-28 03:54:03,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:54:03,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:54:03,990 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:54:03,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:54:03,990 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:03,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:54:03,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-11-28 03:54:03,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-11-28 03:54:03,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:54:03,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-11-28 03:54:03,991 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:03,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:03,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:54:03,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-11-28 03:54:03,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:03,994 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:03,994 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-11-28 03:54:03,994 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-11-28 03:54:03,994 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:03,994 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-11-28 03:54:03,994 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-11-28 03:54:03,994 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:54:03,994 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-11-28 03:54:03,994 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:54:03,994 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:54:03,994 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:54:03,994 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testScannerOnBulkLoadRefHFiles' 2023-11-28 03:54:03,994 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-28 03:54:03,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:54:03,994 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(399): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles because node does not exist (not an error) 2023-11-28 03:54:03,994 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testScannerOnBulkLoadRefHFiles'' to complete. (max 300000 ms per retry) 2023-11-28 03:54:03,995 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-28 03:54:03,995 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:54:03,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:54:04,095 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-28 03:54:04,096 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-28 03:54:04,103 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=38593] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}] of the table testScannerOnBulkLoadRefHFiles 2023-11-28 03:54:04,105 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] procedure2.ProcedureExecutor(1028): Stored pid=34, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testScannerOnBulkLoadRefHFiles.mobLock, type=EXCLUSIVE 2023-11-28 03:54:04,105 DEBUG [PEWorker-5] locking.LockProcedure(309): LOCKED pid=34, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testScannerOnBulkLoadRefHFiles.mobLock, type=EXCLUSIVE 2023-11-28 03:54:04,106 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] mob.MobUtils(833): start MOB compaction of files for table='testScannerOnBulkLoadRefHFiles', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-11-28 03:54:04,106 INFO [PEWorker-5] procedure2.TimeoutExecutorThread(81): ADDED pid=34, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testScannerOnBulkLoadRefHFiles.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1701144244106 2023-11-28 03:54:04,107 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-11-28 03:54:04,107 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 2 selected files, and 0 irrelevant files table 'testScannerOnBulkLoadRefHFiles' and column 'family1' 2023-11-28 03:54:04,107 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testScannerOnBulkLoadRefHFiles' column='family1' 2023-11-28 03:54:04,108 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] zookeeper.ReadOnlyZKClient(139): Connect 0x02451af3 to 127.0.0.1:55641 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-11-28 03:54:04,112 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] ipc.AbstractRpcClient(189): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@6af71db0, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-11-28 03:54:04,113 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c81e728d9d4c2f636f067f89cc14862c20231128 for table='testScannerOnBulkLoadRefHFiles' and column='family1' 2023-11-28 03:54:04,944 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20231128305c9e01167840f4b4ebcf2bf8ee095d to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c20231128305c9e01167840f4b4ebcf2bf8ee095d 2023-11-28 03:54:04,945 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20231128 table='testScannerOnBulkLoadRefHFiles' column='family1' 2023-11-28 03:54:04,965 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(122): Using SIMPLE authentication for service=ClientService, sasl=false 2023-11-28 03:54:04,966 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:55714, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-11-28 03:54:04,973 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(122): Using SIMPLE authentication for service=MasterService, sasl=false 2023-11-28 03:54:04,975 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:43532, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-11-28 03:54:04,984 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testScannerOnBulkLoadRefHFiles/c81e728d9d4c2f636f067f89cc14862c20231128/family1/ef3ac4f2ea7c413095b4791ccfa7eedf first=Optional[k0] last=Optional[k1] 2023-11-28 03:54:04,984 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testScannerOnBulkLoadRefHFiles,2,1701143640845.02be414fee3293f77182043b018e4d6b., hostname=jenkins-hbase4.apache.org,41203,1701143600653, seqNum=2 for row 2 with hfile group [{family1,hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testScannerOnBulkLoadRefHFiles/c81e728d9d4c2f636f067f89cc14862c20231128/family1/ef3ac4f2ea7c413095b4791ccfa7eedf}] 2023-11-28 03:54:04,989 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HStore(763): Validating hfile at hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testScannerOnBulkLoadRefHFiles/c81e728d9d4c2f636f067f89cc14862c20231128/family1/ef3ac4f2ea7c413095b4791ccfa7eedf for inclusion in 02be414fee3293f77182043b018e4d6b/family1 2023-11-28 03:54:04,991 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HStore(775): HFile bounds: first=k0 last=k1 2023-11-28 03:54:04,991 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HStore(777): Region bounds: first=2 last= 2023-11-28 03:54:04,991 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HRegion(2446): Flush status journal for 02be414fee3293f77182043b018e4d6b: 2023-11-28 03:54:04,992 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testScannerOnBulkLoadRefHFiles/c81e728d9d4c2f636f067f89cc14862c20231128/family1/ef3ac4f2ea7c413095b4791ccfa7eedf to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testScannerOnBulkLoadRefHFiles__7mr1gtinjovke1pt13hid7lpns0se17tmbgmer5tq5so072q7dp45fimkvuu02a8/family1/ef3ac4f2ea7c413095b4791ccfa7eedf 2023-11-28 03:54:04,994 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testScannerOnBulkLoadRefHFiles__7mr1gtinjovke1pt13hid7lpns0se17tmbgmer5tq5so072q7dp45fimkvuu02a8/family1/ef3ac4f2ea7c413095b4791ccfa7eedf as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testScannerOnBulkLoadRefHFiles/02be414fee3293f77182043b018e4d6b/family1/b172f96662544ab997303d5df572db53_SeqId_12_ 2023-11-28 03:54:04,995 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HStore(849): Loaded HFile hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testScannerOnBulkLoadRefHFiles__7mr1gtinjovke1pt13hid7lpns0se17tmbgmer5tq5so072q7dp45fimkvuu02a8/family1/ef3ac4f2ea7c413095b4791ccfa7eedf into 02be414fee3293f77182043b018e4d6b/family1 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testScannerOnBulkLoadRefHFiles/02be414fee3293f77182043b018e4d6b/family1/b172f96662544ab997303d5df572db53_SeqId_12_ - updating store file list. 2023-11-28 03:54:05,002 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HStore(882): Loaded HFile hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testScannerOnBulkLoadRefHFiles/02be414fee3293f77182043b018e4d6b/family1/b172f96662544ab997303d5df572db53_SeqId_12_ into 02be414fee3293f77182043b018e4d6b/family1 2023-11-28 03:54:05,002 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HStore(855): Successfully loaded hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testScannerOnBulkLoadRefHFiles__7mr1gtinjovke1pt13hid7lpns0se17tmbgmer5tq5so072q7dp45fimkvuu02a8/family1/ef3ac4f2ea7c413095b4791ccfa7eedf into 02be414fee3293f77182043b018e4d6b/family1 (new location: hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testScannerOnBulkLoadRefHFiles/02be414fee3293f77182043b018e4d6b/family1/b172f96662544ab997303d5df572db53_SeqId_12_) 2023-11-28 03:54:05,002 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testScannerOnBulkLoadRefHFiles__7mr1gtinjovke1pt13hid7lpns0se17tmbgmer5tq5so072q7dp45fimkvuu02a8/family1/ef3ac4f2ea7c413095b4791ccfa7eedf 2023-11-28 03:54:05,002 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-11-28 03:54:05,003 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 15445 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-11-28 03:54:05,003 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HStore(1912): 02be414fee3293f77182043b018e4d6b/family1 is initiating minor compaction (all files) 2023-11-28 03:54:05,003 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@1b43b67d; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-28 03:54:05,003 INFO [RS:0;jenkins-hbase4:41203-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 02be414fee3293f77182043b018e4d6b/family1 in testScannerOnBulkLoadRefHFiles,2,1701143640845.02be414fee3293f77182043b018e4d6b. 2023-11-28 03:54:05,004 INFO [RS:0;jenkins-hbase4:41203-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testScannerOnBulkLoadRefHFiles/02be414fee3293f77182043b018e4d6b/family1/1f7cad3095c64232b280fd227f7cb144, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testScannerOnBulkLoadRefHFiles/02be414fee3293f77182043b018e4d6b/family1/e2b38dc5c6894cda8fb11afc60de22f5, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testScannerOnBulkLoadRefHFiles/02be414fee3293f77182043b018e4d6b/family1/b172f96662544ab997303d5df572db53_SeqId_12_] into tmpdir=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testScannerOnBulkLoadRefHFiles/02be414fee3293f77182043b018e4d6b/.tmp, totalSize=15.1 K 2023-11-28 03:54:05,005 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] compactions.Compactor(207): Compacting 1f7cad3095c64232b280fd227f7cb144, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1701143642955 2023-11-28 03:54:05,005 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] compactions.Compactor(207): Compacting e2b38dc5c6894cda8fb11afc60de22f5, keycount=2, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1701143642955 2023-11-28 03:54:05,006 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] compactions.Compactor(207): Compacting b172f96662544ab997303d5df572db53_SeqId_12_, keycount=2, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1701143642955 2023-11-28 03:54:05,006 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20231128 table='testScannerOnBulkLoadRefHFiles' column='family1' 2023-11-28 03:54:05,007 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-28 03:54:05,008 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c2023112823a282d0bb204bcd87fd14585188d0e9 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c2023112823a282d0bb204bcd87fd14585188d0e9 2023-11-28 03:54:05,010 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c2023112899aa86221a204c73bd94f5f3fe7d5016 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c2023112899aa86221a204c73bd94f5f3fe7d5016 2023-11-28 03:54:05,010 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testScannerOnBulkLoadRefHFiles 2 expired mob files are deleted 2023-11-28 03:54:05,010 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testScannerOnBulkLoadRefHFiles/c81e728d9d4c2f636f067f89cc14862c20231128' 2023-11-28 03:54:05,010 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=c81e728d9d4c2f636f067f89cc14862c20231128 for table='testScannerOnBulkLoadRefHFiles' and column='family1' 2023-11-28 03:54:05,011 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] compactions.PartitionedMobCompactor(354): After compaction, there are 1 mob files. table='testScannerOnBulkLoadRefHFiles' column='family1' 2023-11-28 03:54:05,011 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] mob.MobUtils(840): end MOB compaction of files for table='testScannerOnBulkLoadRefHFiles', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-11-28 03:54:05,012 DEBUG [PEWorker-4] locking.LockProcedure(242): UNLOCKED pid=34, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testScannerOnBulkLoadRefHFiles.mobLock, type=EXCLUSIVE 2023-11-28 03:54:05,013 INFO [PEWorker-4] procedure2.ProcedureExecutor(1409): Finished pid=34, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testScannerOnBulkLoadRefHFiles.mobLock, type=EXCLUSIVE in 908 msec 2023-11-28 03:54:05,020 INFO [RS:0;jenkins-hbase4:41203-shortCompactions-0] throttle.PressureAwareThroughputController(145): 02be414fee3293f77182043b018e4d6b#family1#compaction#37 average throughput is 0.03 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-11-28 03:54:05,045 INFO [Listener at localhost/42853] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testScannerOnBulkLoadRefHFiles Thread=250 (was 261), OpenFileDescriptor=613 (was 651), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=214 (was 224), ProcessCount=172 (was 172), AvailableMemoryMB=6984 (was 7027) 2023-11-28 03:54:05,056 INFO [Listener at localhost/42853] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyPolicy Thread=250, OpenFileDescriptor=613, MaxFileDescriptor=60000, SystemLoadAverage=214, ProcessCount=172, AvailableMemoryMB=6983 2023-11-28 03:54:05,058 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testMajorCompactionWithWeeklyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-11-28 03:54:05,059 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] procedure2.ProcedureExecutor(1028): Stored pid=35, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:05,060 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_PRE_OPERATION 2023-11-28 03:54:05,061 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(700): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionWithWeeklyPolicy" procId is: 35 2023-11-28 03:54:05,061 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-11-28 03:54:05,061 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(1230): Checking to see if procedure is done pid=35 2023-11-28 03:54:05,063 DEBUG [HFileArchiver-8] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/.tmp/data/default/testMajorCompactionWithWeeklyPolicy/8c6dc3114c8460cdf466d087b23e49ef 2023-11-28 03:54:05,063 DEBUG [HFileArchiver-8] backup.HFileArchiver(153): Directory hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/.tmp/data/default/testMajorCompactionWithWeeklyPolicy/8c6dc3114c8460cdf466d087b23e49ef empty. 2023-11-28 03:54:05,064 DEBUG [HFileArchiver-8] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/.tmp/data/default/testMajorCompactionWithWeeklyPolicy/8c6dc3114c8460cdf466d087b23e49ef 2023-11-28 03:54:05,064 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived testMajorCompactionWithWeeklyPolicy regions 2023-11-28 03:54:05,163 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(1230): Checking to see if procedure is done pid=35 2023-11-28 03:54:05,364 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(1230): Checking to see if procedure is done pid=35 2023-11-28 03:54:05,472 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/.tmp/data/default/testMajorCompactionWithWeeklyPolicy/.tabledesc/.tableinfo.0000000001 2023-11-28 03:54:05,474 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(7675): creating {ENCODED => 8c6dc3114c8460cdf466d087b23e49ef, NAME => 'testMajorCompactionWithWeeklyPolicy,,1701143645058.8c6dc3114c8460cdf466d087b23e49ef.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMajorCompactionWithWeeklyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}, regionDir=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/.tmp 2023-11-28 03:54:05,665 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(1230): Checking to see if procedure is done pid=35 2023-11-28 03:54:05,845 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testScannerOnBulkLoadRefHFiles/02be414fee3293f77182043b018e4d6b/.tmp/family1/3c22fb16bfbb42279b4a15a2a2bbbec0 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testScannerOnBulkLoadRefHFiles/02be414fee3293f77182043b018e4d6b/family1/3c22fb16bfbb42279b4a15a2a2bbbec0 2023-11-28 03:54:05,850 INFO [RS:0;jenkins-hbase4:41203-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 02be414fee3293f77182043b018e4d6b/family1 of 02be414fee3293f77182043b018e4d6b into 3c22fb16bfbb42279b4a15a2a2bbbec0(size=5.1 K), total size for store is 5.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-11-28 03:54:05,851 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 02be414fee3293f77182043b018e4d6b: 2023-11-28 03:54:05,851 INFO [RS:0;jenkins-hbase4:41203-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testScannerOnBulkLoadRefHFiles,2,1701143640845.02be414fee3293f77182043b018e4d6b., storeName=family1, fileCount=3, fileSize=15.1 K (4.9 K, 5.0 K, 5.2 K), priority=2, time=1701143645003; duration=0sec 2023-11-28 03:54:05,851 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-28 03:54:05,876 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testScannerOnBulkLoadRefHFiles/02be414fee3293f77182043b018e4d6b/family1/1f7cad3095c64232b280fd227f7cb144, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testScannerOnBulkLoadRefHFiles/02be414fee3293f77182043b018e4d6b/family1/e2b38dc5c6894cda8fb11afc60de22f5, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testScannerOnBulkLoadRefHFiles/02be414fee3293f77182043b018e4d6b/family1/b172f96662544ab997303d5df572db53_SeqId_12_] to archive 2023-11-28 03:54:05,877 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-28 03:54:05,879 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testScannerOnBulkLoadRefHFiles/02be414fee3293f77182043b018e4d6b/family1/1f7cad3095c64232b280fd227f7cb144 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testScannerOnBulkLoadRefHFiles/02be414fee3293f77182043b018e4d6b/family1/1f7cad3095c64232b280fd227f7cb144 2023-11-28 03:54:05,879 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyPolicy,,1701143645058.8c6dc3114c8460cdf466d087b23e49ef.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-28 03:54:05,880 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1604): Closing 8c6dc3114c8460cdf466d087b23e49ef, disabling compactions & flushes 2023-11-28 03:54:05,880 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyPolicy,,1701143645058.8c6dc3114c8460cdf466d087b23e49ef. 2023-11-28 03:54:05,880 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyPolicy,,1701143645058.8c6dc3114c8460cdf466d087b23e49ef. 2023-11-28 03:54:05,880 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyPolicy,,1701143645058.8c6dc3114c8460cdf466d087b23e49ef. after waiting 0 ms 2023-11-28 03:54:05,880 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyPolicy,,1701143645058.8c6dc3114c8460cdf466d087b23e49ef. 2023-11-28 03:54:05,880 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyPolicy,,1701143645058.8c6dc3114c8460cdf466d087b23e49ef. 2023-11-28 03:54:05,880 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1558): Region close journal for 8c6dc3114c8460cdf466d087b23e49ef: 2023-11-28 03:54:05,881 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testScannerOnBulkLoadRefHFiles/02be414fee3293f77182043b018e4d6b/family1/e2b38dc5c6894cda8fb11afc60de22f5 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testScannerOnBulkLoadRefHFiles/02be414fee3293f77182043b018e4d6b/family1/e2b38dc5c6894cda8fb11afc60de22f5 2023-11-28 03:54:05,882 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_ADD_TO_META 2023-11-28 03:54:05,883 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testScannerOnBulkLoadRefHFiles/02be414fee3293f77182043b018e4d6b/family1/b172f96662544ab997303d5df572db53_SeqId_12_ to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testScannerOnBulkLoadRefHFiles/02be414fee3293f77182043b018e4d6b/family1/b172f96662544ab997303d5df572db53_SeqId_12_ 2023-11-28 03:54:05,884 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyPolicy,,1701143645058.8c6dc3114c8460cdf466d087b23e49ef.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1701143645884"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1701143645884"}]},"ts":"1701143645884"} 2023-11-28 03:54:05,885 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-11-28 03:54:05,886 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-11-28 03:54:05,886 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1701143645886"}]},"ts":"1701143645886"} 2023-11-28 03:54:05,887 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyPolicy, state=ENABLING in hbase:meta 2023-11-28 03:54:05,891 INFO [PEWorker-3] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyPolicy, region=8c6dc3114c8460cdf466d087b23e49ef, ASSIGN}] 2023-11-28 03:54:05,893 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyPolicy, region=8c6dc3114c8460cdf466d087b23e49ef, ASSIGN 2023-11-28 03:54:05,893 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyPolicy, region=8c6dc3114c8460cdf466d087b23e49ef, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,41203,1701143600653; forceNewPlan=false, retain=false 2023-11-28 03:54:06,044 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=36 updating hbase:meta row=8c6dc3114c8460cdf466d087b23e49ef, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:06,045 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyPolicy,,1701143645058.8c6dc3114c8460cdf466d087b23e49ef.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1701143646044"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1701143646044"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1701143646044"}]},"ts":"1701143646044"} 2023-11-28 03:54:06,046 INFO [PEWorker-2] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=37, ppid=36, state=RUNNABLE; OpenRegionProcedure 8c6dc3114c8460cdf466d087b23e49ef, server=jenkins-hbase4.apache.org,41203,1701143600653}] 2023-11-28 03:54:06,166 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(1230): Checking to see if procedure is done pid=35 2023-11-28 03:54:06,198 DEBUG [RSProcedureDispatcher-pool-2] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:06,201 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testMajorCompactionWithWeeklyPolicy,,1701143645058.8c6dc3114c8460cdf466d087b23e49ef. 2023-11-28 03:54:06,201 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 8c6dc3114c8460cdf466d087b23e49ef, NAME => 'testMajorCompactionWithWeeklyPolicy,,1701143645058.8c6dc3114c8460cdf466d087b23e49ef.', STARTKEY => '', ENDKEY => ''} 2023-11-28 03:54:06,202 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyPolicy 8c6dc3114c8460cdf466d087b23e49ef 2023-11-28 03:54:06,202 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyPolicy,,1701143645058.8c6dc3114c8460cdf466d087b23e49ef.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-28 03:54:06,202 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 8c6dc3114c8460cdf466d087b23e49ef 2023-11-28 03:54:06,202 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 8c6dc3114c8460cdf466d087b23e49ef 2023-11-28 03:54:06,203 INFO [StoreOpener-8c6dc3114c8460cdf466d087b23e49ef-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 8c6dc3114c8460cdf466d087b23e49ef 2023-11-28 03:54:06,204 DEBUG [StoreOpener-8c6dc3114c8460cdf466d087b23e49ef-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyPolicy/8c6dc3114c8460cdf466d087b23e49ef/family1 2023-11-28 03:54:06,204 DEBUG [StoreOpener-8c6dc3114c8460cdf466d087b23e49ef-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyPolicy/8c6dc3114c8460cdf466d087b23e49ef/family1 2023-11-28 03:54:06,205 INFO [StoreOpener-8c6dc3114c8460cdf466d087b23e49ef-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 8c6dc3114c8460cdf466d087b23e49ef columnFamilyName family1 2023-11-28 03:54:06,206 INFO [StoreOpener-8c6dc3114c8460cdf466d087b23e49ef-1] regionserver.HStore(310): Store=8c6dc3114c8460cdf466d087b23e49ef/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-28 03:54:06,207 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyPolicy/8c6dc3114c8460cdf466d087b23e49ef 2023-11-28 03:54:06,208 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyPolicy/8c6dc3114c8460cdf466d087b23e49ef 2023-11-28 03:54:06,210 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 8c6dc3114c8460cdf466d087b23e49ef 2023-11-28 03:54:06,212 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyPolicy/8c6dc3114c8460cdf466d087b23e49ef/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-11-28 03:54:06,213 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 8c6dc3114c8460cdf466d087b23e49ef; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60282446, jitterRate=-0.10172155499458313}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-11-28 03:54:06,213 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 8c6dc3114c8460cdf466d087b23e49ef: 2023-11-28 03:54:06,214 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2339): Post open deploy tasks for testMajorCompactionWithWeeklyPolicy,,1701143645058.8c6dc3114c8460cdf466d087b23e49ef., pid=37, masterSystemTime=1701143646198 2023-11-28 03:54:06,215 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2366): Finished post open deploy task for testMajorCompactionWithWeeklyPolicy,,1701143645058.8c6dc3114c8460cdf466d087b23e49ef. 2023-11-28 03:54:06,216 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testMajorCompactionWithWeeklyPolicy,,1701143645058.8c6dc3114c8460cdf466d087b23e49ef. 2023-11-28 03:54:06,216 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=36 updating hbase:meta row=8c6dc3114c8460cdf466d087b23e49ef, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:06,216 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyPolicy,,1701143645058.8c6dc3114c8460cdf466d087b23e49ef.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1701143646216"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1701143646216"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1701143646216"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1701143646216"}]},"ts":"1701143646216"} 2023-11-28 03:54:06,219 INFO [PEWorker-4] procedure2.ProcedureExecutor(1825): Finished subprocedure pid=37, resume processing ppid=36 2023-11-28 03:54:06,219 INFO [PEWorker-4] procedure2.ProcedureExecutor(1409): Finished pid=37, ppid=36, state=SUCCESS; OpenRegionProcedure 8c6dc3114c8460cdf466d087b23e49ef, server=jenkins-hbase4.apache.org,41203,1701143600653 in 171 msec 2023-11-28 03:54:06,221 INFO [PEWorker-3] procedure2.ProcedureExecutor(1825): Finished subprocedure pid=36, resume processing ppid=35 2023-11-28 03:54:06,221 INFO [PEWorker-3] procedure2.ProcedureExecutor(1409): Finished pid=36, ppid=35, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyPolicy, region=8c6dc3114c8460cdf466d087b23e49ef, ASSIGN in 328 msec 2023-11-28 03:54:06,221 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-11-28 03:54:06,222 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1701143646222"}]},"ts":"1701143646222"} 2023-11-28 03:54:06,223 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyPolicy, state=ENABLED in hbase:meta 2023-11-28 03:54:06,238 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_POST_OPERATION 2023-11-28 03:54:06,240 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=35, state=SUCCESS; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy in 1.1810 sec 2023-11-28 03:54:07,167 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(1230): Checking to see if procedure is done pid=35 2023-11-28 03:54:07,168 INFO [Listener at localhost/42853] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionWithWeeklyPolicy, procId: 35 completed 2023-11-28 03:54:07,197 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-28 03:54:07,198 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:07,198 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-11-28 03:54:07,198 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:54:07,199 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-11-28 03:54:07,199 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-11-28 03:54:07,199 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:07,199 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:07,201 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:54:07,201 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:07,201 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:54:07,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:54:07,201 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:07,201 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-28 03:54:07,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:07,201 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:07,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-28 03:54:07,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:07,202 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:07,202 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:07,202 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-11-28 03:54:07,202 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:54:07,202 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-11-28 03:54:07,202 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-28 03:54:07,202 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-28 03:54:07,202 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1701143645058.8c6dc3114c8460cdf466d087b23e49ef. 2023-11-28 03:54:07,202 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1701143645058.8c6dc3114c8460cdf466d087b23e49ef. started... 2023-11-28 03:54:07,203 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 8c6dc3114c8460cdf466d087b23e49ef 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-11-28 03:54:07,568 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-11-28 03:54:07,626 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015113004e9084677c1492894955371e22862a6 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2015113004e9084677c1492894955371e22862a6 2023-11-28 03:54:08,030 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyPolicy/8c6dc3114c8460cdf466d087b23e49ef/.tmp/family1/299aee69722249a898d53379abc8c8c3 2023-11-28 03:54:08,035 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyPolicy/8c6dc3114c8460cdf466d087b23e49ef/.tmp/family1/299aee69722249a898d53379abc8c8c3 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyPolicy/8c6dc3114c8460cdf466d087b23e49ef/family1/299aee69722249a898d53379abc8c8c3 2023-11-28 03:54:08,040 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyPolicy/8c6dc3114c8460cdf466d087b23e49ef/family1/299aee69722249a898d53379abc8c8c3, entries=1000, sequenceid=5, filesize=165.1 K 2023-11-28 03:54:08,041 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for 8c6dc3114c8460cdf466d087b23e49ef in 838ms, sequenceid=5, compaction requested=false 2023-11-28 03:54:08,041 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithWeeklyPolicy' 2023-11-28 03:54:08,042 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 8c6dc3114c8460cdf466d087b23e49ef: 2023-11-28 03:54:08,042 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1701143645058.8c6dc3114c8460cdf466d087b23e49ef. 2023-11-28 03:54:08,042 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-28 03:54:08,042 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-28 03:54:08,042 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:08,042 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-11-28 03:54:08,042 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-11-28 03:54:08,044 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:08,044 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:08,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:08,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:54:08,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:54:08,044 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:08,044 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-28 03:54:08,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:54:08,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:54:08,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:08,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:08,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:54:08,046 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-11-28 03:54:08,046 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2e8d35b0[Count = 0] remaining members to acquire global barrier 2023-11-28 03:54:08,046 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-11-28 03:54:08,046 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:08,048 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:08,048 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:08,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:08,048 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-11-28 03:54:08,048 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-11-28 03:54:08,048 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,41203,1701143600653' in zk 2023-11-28 03:54:08,048 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:08,048 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-28 03:54:08,049 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:08,049 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-11-28 03:54:08,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:08,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:54:08,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:54:08,050 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:54:08,050 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-11-28 03:54:08,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:54:08,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:54:08,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:08,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:08,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:54:08,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:08,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:08,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,41203,1701143600653': 2023-11-28 03:54:08,052 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-11-28 03:54:08,052 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-11-28 03:54:08,052 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-28 03:54:08,052 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-28 03:54:08,052 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:08,052 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-28 03:54:08,054 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:08,054 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:08,054 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:54:08,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:08,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:54:08,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:54:08,054 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:08,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:08,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:54:08,054 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:54:08,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:54:08,054 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:08,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:08,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:08,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:54:08,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:08,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:08,056 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:08,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:54:08,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:08,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:08,058 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:08,058 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:54:08,058 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:08,058 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:54:08,058 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:08,058 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:54:08,058 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:54:08,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:54:08,058 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-11-28 03:54:08,058 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-28 03:54:08,058 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:08,058 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:08,058 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:08,058 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:08,058 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:54:08,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:54:08,059 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-11-28 03:54:08,059 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-28 03:54:08,159 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-28 03:54:08,160 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-28 03:54:08,164 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-28 03:54:08,166 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-11-28 03:54:08,166 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:08,166 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-11-28 03:54:08,166 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:54:08,166 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-11-28 03:54:08,166 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-11-28 03:54:08,167 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:08,167 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:08,169 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:54:08,169 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:08,169 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:54:08,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:54:08,169 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:08,169 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-28 03:54:08,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:08,170 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:08,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-28 03:54:08,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:08,170 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:08,170 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-11-28 03:54:08,170 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:08,170 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-11-28 03:54:08,170 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:54:08,170 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-11-28 03:54:08,171 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-28 03:54:08,171 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-28 03:54:08,171 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1701143645058.8c6dc3114c8460cdf466d087b23e49ef. 2023-11-28 03:54:08,171 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1701143645058.8c6dc3114c8460cdf466d087b23e49ef. started... 2023-11-28 03:54:08,171 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 8c6dc3114c8460cdf466d087b23e49ef 1/1 column families, dataSize=74 B heapSize=408 B 2023-11-28 03:54:08,584 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128b4df4a9306264aba89d7757563962166 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151128b4df4a9306264aba89d7757563962166 2023-11-28 03:54:08,990 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyPolicy/8c6dc3114c8460cdf466d087b23e49ef/.tmp/family1/967f2a3befe74ada91cd38df5a1f2d39 2023-11-28 03:54:08,995 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyPolicy/8c6dc3114c8460cdf466d087b23e49ef/.tmp/family1/967f2a3befe74ada91cd38df5a1f2d39 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyPolicy/8c6dc3114c8460cdf466d087b23e49ef/family1/967f2a3befe74ada91cd38df5a1f2d39 2023-11-28 03:54:08,999 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyPolicy/8c6dc3114c8460cdf466d087b23e49ef/family1/967f2a3befe74ada91cd38df5a1f2d39, entries=1, sequenceid=9, filesize=5.0 K 2023-11-28 03:54:09,000 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for 8c6dc3114c8460cdf466d087b23e49ef in 829ms, sequenceid=9, compaction requested=false 2023-11-28 03:54:09,000 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 8c6dc3114c8460cdf466d087b23e49ef: 2023-11-28 03:54:09,000 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1701143645058.8c6dc3114c8460cdf466d087b23e49ef. 2023-11-28 03:54:09,000 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-28 03:54:09,001 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-28 03:54:09,001 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:09,001 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-11-28 03:54:09,001 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-11-28 03:54:09,002 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:09,002 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:09,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:09,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:54:09,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:54:09,003 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:09,003 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-28 03:54:09,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:54:09,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:54:09,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:09,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:09,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:54:09,004 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-11-28 03:54:09,004 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7edcd355[Count = 0] remaining members to acquire global barrier 2023-11-28 03:54:09,004 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-11-28 03:54:09,004 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:09,006 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:09,006 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:09,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:09,006 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-11-28 03:54:09,006 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-11-28 03:54:09,006 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,41203,1701143600653' in zk 2023-11-28 03:54:09,006 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:09,006 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-28 03:54:09,007 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:09,007 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-11-28 03:54:09,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:09,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:54:09,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:54:09,008 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:54:09,008 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-11-28 03:54:09,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:54:09,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:54:09,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:09,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:09,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:54:09,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:09,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:09,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,41203,1701143600653': 2023-11-28 03:54:09,010 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-11-28 03:54:09,010 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-11-28 03:54:09,010 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-28 03:54:09,010 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-28 03:54:09,010 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:09,010 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-28 03:54:09,013 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:09,013 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:09,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:09,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:54:09,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:54:09,013 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:09,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:09,013 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:54:09,013 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:09,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:54:09,013 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:54:09,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:54:09,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:09,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:09,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:54:09,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:09,014 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:09,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:09,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:54:09,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:09,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:09,019 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:09,019 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:54:09,019 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:09,020 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:54:09,020 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:09,020 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:09,020 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:54:09,019 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:54:09,020 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-11-28 03:54:09,020 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-28 03:54:09,020 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:09,020 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(399): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-11-28 03:54:09,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:54:09,020 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:09,020 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:09,020 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-11-28 03:54:09,020 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-28 03:54:09,020 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:54:09,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:54:09,120 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-28 03:54:09,121 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-28 03:54:09,126 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-28 03:54:09,127 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-11-28 03:54:09,127 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:09,127 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-11-28 03:54:09,128 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:54:09,128 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-11-28 03:54:09,128 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-11-28 03:54:09,128 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:09,128 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:09,130 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:54:09,130 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:09,130 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:54:09,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:54:09,130 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:09,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:09,130 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-28 03:54:09,130 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:09,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-28 03:54:09,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:09,131 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:09,131 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-11-28 03:54:09,131 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:09,131 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-11-28 03:54:09,131 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:54:09,131 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-11-28 03:54:09,131 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-28 03:54:09,131 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-28 03:54:09,131 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1701143645058.8c6dc3114c8460cdf466d087b23e49ef. 2023-11-28 03:54:09,131 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1701143645058.8c6dc3114c8460cdf466d087b23e49ef. started... 2023-11-28 03:54:09,131 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 8c6dc3114c8460cdf466d087b23e49ef 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-28 03:54:09,143 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151201f5b083ae53594740b7bd7b5806bf03ee to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151201f5b083ae53594740b7bd7b5806bf03ee 2023-11-28 03:54:09,549 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyPolicy/8c6dc3114c8460cdf466d087b23e49ef/.tmp/family1/423d77dca0d54cca89f8eb673fb69407 2023-11-28 03:54:09,554 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyPolicy/8c6dc3114c8460cdf466d087b23e49ef/.tmp/family1/423d77dca0d54cca89f8eb673fb69407 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyPolicy/8c6dc3114c8460cdf466d087b23e49ef/family1/423d77dca0d54cca89f8eb673fb69407 2023-11-28 03:54:09,558 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyPolicy/8c6dc3114c8460cdf466d087b23e49ef/family1/423d77dca0d54cca89f8eb673fb69407, entries=1, sequenceid=13, filesize=4.9 K 2023-11-28 03:54:09,559 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 8c6dc3114c8460cdf466d087b23e49ef in 428ms, sequenceid=13, compaction requested=true 2023-11-28 03:54:09,559 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 8c6dc3114c8460cdf466d087b23e49ef: 2023-11-28 03:54:09,559 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1701143645058.8c6dc3114c8460cdf466d087b23e49ef. 2023-11-28 03:54:09,560 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-28 03:54:09,560 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-28 03:54:09,560 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:09,560 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-11-28 03:54:09,560 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-11-28 03:54:09,562 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:09,562 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:09,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:09,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:54:09,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:54:09,563 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:09,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:54:09,563 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-28 03:54:09,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:54:09,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:09,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:09,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:54:09,564 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-11-28 03:54:09,564 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@400ca479[Count = 0] remaining members to acquire global barrier 2023-11-28 03:54:09,564 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-11-28 03:54:09,564 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:09,566 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:09,566 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:09,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:09,567 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-11-28 03:54:09,567 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-11-28 03:54:09,567 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,41203,1701143600653' in zk 2023-11-28 03:54:09,567 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:09,567 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-28 03:54:09,568 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-11-28 03:54:09,568 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:09,568 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:54:09,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:09,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:54:09,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:54:09,568 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-11-28 03:54:09,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:54:09,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:54:09,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:09,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:09,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:54:09,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:09,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:09,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,41203,1701143600653': 2023-11-28 03:54:09,571 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-11-28 03:54:09,571 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-11-28 03:54:09,571 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-28 03:54:09,571 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-28 03:54:09,571 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:09,571 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-28 03:54:09,574 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:09,574 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:09,574 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:09,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:09,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:54:09,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:54:09,574 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:54:09,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:09,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:54:09,575 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:54:09,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:54:09,575 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:09,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:09,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:09,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:54:09,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:09,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:09,576 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:09,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:54:09,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:09,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:09,579 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:09,579 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:54:09,579 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:09,579 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:54:09,579 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:54:09,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:54:09,580 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-11-28 03:54:09,580 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-28 03:54:09,580 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:54:09,579 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:09,580 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:09,580 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:09,580 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:09,580 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:54:09,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:54:09,580 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:09,580 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-11-28 03:54:09,580 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-28 03:54:09,680 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-28 03:54:09,681 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-28 03:54:09,686 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-28 03:54:09,687 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-11-28 03:54:09,687 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:09,687 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-11-28 03:54:09,687 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:54:09,688 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-11-28 03:54:09,688 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-11-28 03:54:09,688 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:09,688 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:09,689 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:09,690 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:54:09,690 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:54:09,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:54:09,690 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:09,690 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-28 03:54:09,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:09,690 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:09,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-28 03:54:09,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:09,691 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:09,691 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-11-28 03:54:09,691 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:09,691 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-11-28 03:54:09,691 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:54:09,691 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-11-28 03:54:09,691 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-28 03:54:09,691 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-28 03:54:09,691 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1701143645058.8c6dc3114c8460cdf466d087b23e49ef. 2023-11-28 03:54:09,691 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1701143645058.8c6dc3114c8460cdf466d087b23e49ef. started... 2023-11-28 03:54:09,691 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 8c6dc3114c8460cdf466d087b23e49ef 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-28 03:54:10,105 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205833d85e416cd4a018dd0d85b4759ad73 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151205833d85e416cd4a018dd0d85b4759ad73 2023-11-28 03:54:10,511 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyPolicy/8c6dc3114c8460cdf466d087b23e49ef/.tmp/family1/71b79bd8528445379ad8504e5a0108a4 2023-11-28 03:54:10,516 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyPolicy/8c6dc3114c8460cdf466d087b23e49ef/.tmp/family1/71b79bd8528445379ad8504e5a0108a4 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyPolicy/8c6dc3114c8460cdf466d087b23e49ef/family1/71b79bd8528445379ad8504e5a0108a4 2023-11-28 03:54:10,521 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyPolicy/8c6dc3114c8460cdf466d087b23e49ef/family1/71b79bd8528445379ad8504e5a0108a4, entries=1, sequenceid=17, filesize=4.9 K 2023-11-28 03:54:10,522 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 8c6dc3114c8460cdf466d087b23e49ef in 831ms, sequenceid=17, compaction requested=true 2023-11-28 03:54:10,522 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 8c6dc3114c8460cdf466d087b23e49ef: 2023-11-28 03:54:10,522 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1701143645058.8c6dc3114c8460cdf466d087b23e49ef. 2023-11-28 03:54:10,522 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-28 03:54:10,522 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-28 03:54:10,522 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:10,522 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-11-28 03:54:10,522 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-11-28 03:54:10,524 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:10,524 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:10,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:10,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:54:10,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:54:10,525 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:10,525 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-28 03:54:10,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:54:10,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:54:10,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:10,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:10,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:54:10,526 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-11-28 03:54:10,526 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@614f7015[Count = 0] remaining members to acquire global barrier 2023-11-28 03:54:10,526 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-11-28 03:54:10,526 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:10,527 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:10,527 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:10,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:10,527 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-11-28 03:54:10,528 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-11-28 03:54:10,528 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,41203,1701143600653' in zk 2023-11-28 03:54:10,528 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:10,528 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-28 03:54:10,530 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:10,530 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-11-28 03:54:10,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:10,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:54:10,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:54:10,530 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:54:10,530 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-11-28 03:54:10,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:54:10,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:54:10,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:10,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:10,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:54:10,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:10,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:10,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,41203,1701143600653': 2023-11-28 03:54:10,533 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-11-28 03:54:10,533 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-11-28 03:54:10,533 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-28 03:54:10,533 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-28 03:54:10,533 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:10,533 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-28 03:54:10,534 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:10,534 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:54:10,534 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:10,534 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:10,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:10,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:10,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:54:10,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:54:10,535 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:54:10,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:54:10,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:54:10,535 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:10,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:10,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:10,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:54:10,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:10,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:10,537 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:10,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:54:10,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:10,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:10,539 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:10,539 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:54:10,539 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:10,539 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:54:10,539 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-11-28 03:54:10,539 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-28 03:54:10,539 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:54:10,539 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:10,539 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:54:10,540 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:10,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:54:10,540 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:10,540 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:10,540 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:10,540 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-11-28 03:54:10,540 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-28 03:54:10,540 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:54:10,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:54:10,640 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-28 03:54:10,641 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-28 03:54:10,645 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-28 03:54:10,646 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-11-28 03:54:10,646 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:10,646 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-11-28 03:54:10,646 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:54:10,646 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-11-28 03:54:10,646 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-11-28 03:54:10,647 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:10,647 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:10,648 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:54:10,648 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:10,648 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:54:10,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:54:10,649 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:10,649 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-28 03:54:10,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:10,649 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:10,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-28 03:54:10,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:10,649 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:10,650 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-11-28 03:54:10,650 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:10,650 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-11-28 03:54:10,650 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:54:10,650 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-11-28 03:54:10,650 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-28 03:54:10,650 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-28 03:54:10,650 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1701143645058.8c6dc3114c8460cdf466d087b23e49ef. 2023-11-28 03:54:10,650 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1701143645058.8c6dc3114c8460cdf466d087b23e49ef. started... 2023-11-28 03:54:10,650 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 8c6dc3114c8460cdf466d087b23e49ef 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-28 03:54:11,063 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151228a32717d3745249039b91112b33480e17 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151228a32717d3745249039b91112b33480e17 2023-11-28 03:54:11,470 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyPolicy/8c6dc3114c8460cdf466d087b23e49ef/.tmp/family1/7b192f29e2a248d49efa66a8fbb93b8a 2023-11-28 03:54:11,475 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyPolicy/8c6dc3114c8460cdf466d087b23e49ef/.tmp/family1/7b192f29e2a248d49efa66a8fbb93b8a as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyPolicy/8c6dc3114c8460cdf466d087b23e49ef/family1/7b192f29e2a248d49efa66a8fbb93b8a 2023-11-28 03:54:11,479 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyPolicy/8c6dc3114c8460cdf466d087b23e49ef/family1/7b192f29e2a248d49efa66a8fbb93b8a, entries=1, sequenceid=21, filesize=4.9 K 2023-11-28 03:54:11,480 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 8c6dc3114c8460cdf466d087b23e49ef in 830ms, sequenceid=21, compaction requested=true 2023-11-28 03:54:11,480 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 8c6dc3114c8460cdf466d087b23e49ef: 2023-11-28 03:54:11,480 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1701143645058.8c6dc3114c8460cdf466d087b23e49ef. 2023-11-28 03:54:11,480 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-28 03:54:11,480 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-28 03:54:11,481 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:11,481 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-11-28 03:54:11,481 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-11-28 03:54:11,484 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:11,484 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:11,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:11,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:54:11,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:54:11,484 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:11,484 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-28 03:54:11,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:54:11,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:54:11,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:11,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:11,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:54:11,486 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-11-28 03:54:11,486 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2628f939[Count = 0] remaining members to acquire global barrier 2023-11-28 03:54:11,486 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-11-28 03:54:11,486 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:11,487 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:11,487 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:11,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:11,487 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-11-28 03:54:11,487 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-11-28 03:54:11,487 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,41203,1701143600653' in zk 2023-11-28 03:54:11,487 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:11,487 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-28 03:54:11,489 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:11,489 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-11-28 03:54:11,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:11,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:54:11,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:54:11,489 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:54:11,489 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-11-28 03:54:11,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:54:11,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:54:11,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:11,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:11,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:54:11,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:11,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:11,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,41203,1701143600653': 2023-11-28 03:54:11,491 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-11-28 03:54:11,491 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-11-28 03:54:11,491 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-28 03:54:11,491 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-28 03:54:11,491 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:11,491 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-28 03:54:11,497 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:11,497 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:11,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:11,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:54:11,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:54:11,497 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:11,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:11,497 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:54:11,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:54:11,497 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:54:11,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:54:11,497 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:11,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:11,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:11,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:54:11,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:11,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:11,499 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:11,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:54:11,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:11,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:11,502 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:11,502 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:54:11,502 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:11,502 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:54:11,502 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:11,502 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:11,502 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:54:11,502 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:11,502 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:11,502 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:11,502 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:54:11,502 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-11-28 03:54:11,502 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-28 03:54:11,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:54:11,502 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(399): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-11-28 03:54:11,502 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-11-28 03:54:11,502 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:54:11,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:54:11,502 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-28 03:54:11,603 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-28 03:54:11,603 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-28 03:54:11,607 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-28 03:54:11,609 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-11-28 03:54:11,609 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:11,609 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-11-28 03:54:11,609 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:54:11,609 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-11-28 03:54:11,609 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-11-28 03:54:11,610 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:11,610 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:11,611 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:54:11,611 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:11,611 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:54:11,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:54:11,612 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:11,612 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-28 03:54:11,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:11,612 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:11,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-28 03:54:11,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:11,612 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:11,613 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-11-28 03:54:11,613 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:11,613 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-11-28 03:54:11,613 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:54:11,613 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-11-28 03:54:11,613 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-28 03:54:11,613 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-28 03:54:11,613 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1701143645058.8c6dc3114c8460cdf466d087b23e49ef. 2023-11-28 03:54:11,613 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1701143645058.8c6dc3114c8460cdf466d087b23e49ef. started... 2023-11-28 03:54:11,613 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 8c6dc3114c8460cdf466d087b23e49ef 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-28 03:54:12,026 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151231e5b27507111c445e87d3eb16d0bba565 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151231e5b27507111c445e87d3eb16d0bba565 2023-11-28 03:54:12,031 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyPolicy/8c6dc3114c8460cdf466d087b23e49ef/.tmp/family1/e9b9e857714947d3a2eb0e3a948b1864 2023-11-28 03:54:12,035 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyPolicy/8c6dc3114c8460cdf466d087b23e49ef/.tmp/family1/e9b9e857714947d3a2eb0e3a948b1864 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyPolicy/8c6dc3114c8460cdf466d087b23e49ef/family1/e9b9e857714947d3a2eb0e3a948b1864 2023-11-28 03:54:12,040 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyPolicy/8c6dc3114c8460cdf466d087b23e49ef/family1/e9b9e857714947d3a2eb0e3a948b1864, entries=1, sequenceid=25, filesize=4.9 K 2023-11-28 03:54:12,041 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 8c6dc3114c8460cdf466d087b23e49ef in 428ms, sequenceid=25, compaction requested=true 2023-11-28 03:54:12,041 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 8c6dc3114c8460cdf466d087b23e49ef: 2023-11-28 03:54:12,041 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1701143645058.8c6dc3114c8460cdf466d087b23e49ef. 2023-11-28 03:54:12,041 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-28 03:54:12,041 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-28 03:54:12,041 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:12,041 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-11-28 03:54:12,041 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-11-28 03:54:12,043 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:12,043 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:12,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:12,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:54:12,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:54:12,043 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:12,043 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-28 03:54:12,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:54:12,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:54:12,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:12,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:12,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:54:12,045 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-11-28 03:54:12,045 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3e76d0d2[Count = 0] remaining members to acquire global barrier 2023-11-28 03:54:12,045 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-11-28 03:54:12,045 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:12,047 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:12,047 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:12,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:12,047 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-11-28 03:54:12,047 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-11-28 03:54:12,047 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,41203,1701143600653' in zk 2023-11-28 03:54:12,047 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:12,047 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-28 03:54:12,048 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-11-28 03:54:12,048 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:12,048 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:54:12,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:12,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:54:12,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:54:12,048 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-11-28 03:54:12,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:54:12,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:54:12,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:12,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:12,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:54:12,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:12,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:12,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,41203,1701143600653': 2023-11-28 03:54:12,051 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-11-28 03:54:12,051 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-11-28 03:54:12,051 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-28 03:54:12,051 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-28 03:54:12,051 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:12,051 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-28 03:54:12,053 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:12,053 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:12,053 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:12,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:12,053 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:54:12,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:12,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:54:12,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:54:12,053 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:54:12,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:54:12,053 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:12,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:54:12,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:12,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:12,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:54:12,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:12,055 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:12,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:12,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:54:12,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:12,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:12,057 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:12,057 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:54:12,057 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:12,057 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:54:12,057 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:12,057 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:12,057 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:12,057 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-11-28 03:54:12,057 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-28 03:54:12,057 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:54:12,057 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:12,057 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(399): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-11-28 03:54:12,057 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:54:12,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:54:12,057 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-11-28 03:54:12,057 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:12,058 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-28 03:54:12,058 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:54:12,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:54:12,158 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-28 03:54:12,159 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-28 03:54:12,163 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-28 03:54:12,164 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-11-28 03:54:12,164 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:12,164 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-11-28 03:54:12,164 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:54:12,164 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-11-28 03:54:12,164 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-11-28 03:54:12,165 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:12,165 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:12,167 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:54:12,167 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:12,167 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:54:12,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:54:12,168 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:12,168 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-28 03:54:12,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:12,168 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:12,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-28 03:54:12,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:12,168 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:12,169 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-11-28 03:54:12,169 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:12,169 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-11-28 03:54:12,169 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:54:12,169 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-11-28 03:54:12,169 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-28 03:54:12,169 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-28 03:54:12,169 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1701143645058.8c6dc3114c8460cdf466d087b23e49ef. 2023-11-28 03:54:12,169 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1701143645058.8c6dc3114c8460cdf466d087b23e49ef. started... 2023-11-28 03:54:12,169 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 8c6dc3114c8460cdf466d087b23e49ef 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-28 03:54:12,583 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160101845a31f1b34444f993da6b8d4b7475a8 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20160101845a31f1b34444f993da6b8d4b7475a8 2023-11-28 03:54:12,989 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyPolicy/8c6dc3114c8460cdf466d087b23e49ef/.tmp/family1/bd8c8840b6f3499291fc5e07644dac99 2023-11-28 03:54:12,994 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyPolicy/8c6dc3114c8460cdf466d087b23e49ef/.tmp/family1/bd8c8840b6f3499291fc5e07644dac99 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyPolicy/8c6dc3114c8460cdf466d087b23e49ef/family1/bd8c8840b6f3499291fc5e07644dac99 2023-11-28 03:54:12,998 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyPolicy/8c6dc3114c8460cdf466d087b23e49ef/family1/bd8c8840b6f3499291fc5e07644dac99, entries=1, sequenceid=29, filesize=4.9 K 2023-11-28 03:54:12,999 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 8c6dc3114c8460cdf466d087b23e49ef in 830ms, sequenceid=29, compaction requested=true 2023-11-28 03:54:12,999 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 8c6dc3114c8460cdf466d087b23e49ef: 2023-11-28 03:54:12,999 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1701143645058.8c6dc3114c8460cdf466d087b23e49ef. 2023-11-28 03:54:12,999 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-28 03:54:12,999 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-28 03:54:12,999 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:13,000 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-11-28 03:54:13,000 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-11-28 03:54:13,002 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:13,002 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:13,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:13,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:54:13,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:54:13,002 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:13,002 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-28 03:54:13,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:54:13,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:54:13,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:13,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:13,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:54:13,003 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-11-28 03:54:13,003 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1b7c8409[Count = 0] remaining members to acquire global barrier 2023-11-28 03:54:13,003 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-11-28 03:54:13,004 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:13,005 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:13,005 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:13,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:13,005 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-11-28 03:54:13,005 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-11-28 03:54:13,005 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,41203,1701143600653' in zk 2023-11-28 03:54:13,005 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:13,005 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-28 03:54:13,006 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:13,006 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-11-28 03:54:13,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:13,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:54:13,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:54:13,006 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:54:13,007 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-11-28 03:54:13,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:54:13,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:54:13,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:13,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:13,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:54:13,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:13,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:13,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,41203,1701143600653': 2023-11-28 03:54:13,009 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-11-28 03:54:13,009 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-11-28 03:54:13,009 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-28 03:54:13,009 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-28 03:54:13,009 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:13,009 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-28 03:54:13,011 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:13,011 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:13,011 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:13,011 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:54:13,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:13,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:13,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:54:13,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:54:13,012 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:54:13,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:54:13,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:54:13,012 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:13,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:13,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:13,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:54:13,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:13,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:13,013 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:13,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:54:13,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:13,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:13,016 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:13,016 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:54:13,016 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:13,016 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:54:13,016 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:54:13,016 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:54:13,016 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-11-28 03:54:13,016 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-28 03:54:13,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:54:13,016 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:13,016 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:13,016 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:13,016 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:13,016 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:13,016 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-11-28 03:54:13,017 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-28 03:54:13,017 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:54:13,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:54:13,117 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-28 03:54:13,118 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-28 03:54:13,122 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-28 03:54:13,123 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-11-28 03:54:13,123 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:13,123 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-11-28 03:54:13,123 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:54:13,123 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-11-28 03:54:13,123 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-11-28 03:54:13,124 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:13,124 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:13,125 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:54:13,125 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:13,126 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:54:13,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:54:13,126 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:13,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:13,126 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-28 03:54:13,126 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:13,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-28 03:54:13,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:13,127 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:13,127 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-11-28 03:54:13,127 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:13,127 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-11-28 03:54:13,127 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:54:13,127 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-11-28 03:54:13,127 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-28 03:54:13,127 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-28 03:54:13,127 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1701143645058.8c6dc3114c8460cdf466d087b23e49ef. 2023-11-28 03:54:13,127 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1701143645058.8c6dc3114c8460cdf466d087b23e49ef. started... 2023-11-28 03:54:13,127 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 8c6dc3114c8460cdf466d087b23e49ef 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-28 03:54:13,541 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2016010322855a10b7164ee899d0737e16f47433 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2016010322855a10b7164ee899d0737e16f47433 2023-11-28 03:54:13,946 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyPolicy/8c6dc3114c8460cdf466d087b23e49ef/.tmp/family1/5017517f4d2846a0bc186136ffa4bd5b 2023-11-28 03:54:13,951 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyPolicy/8c6dc3114c8460cdf466d087b23e49ef/.tmp/family1/5017517f4d2846a0bc186136ffa4bd5b as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyPolicy/8c6dc3114c8460cdf466d087b23e49ef/family1/5017517f4d2846a0bc186136ffa4bd5b 2023-11-28 03:54:13,955 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyPolicy/8c6dc3114c8460cdf466d087b23e49ef/family1/5017517f4d2846a0bc186136ffa4bd5b, entries=1, sequenceid=33, filesize=4.9 K 2023-11-28 03:54:13,956 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 8c6dc3114c8460cdf466d087b23e49ef in 829ms, sequenceid=33, compaction requested=true 2023-11-28 03:54:13,956 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 8c6dc3114c8460cdf466d087b23e49ef: 2023-11-28 03:54:13,956 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1701143645058.8c6dc3114c8460cdf466d087b23e49ef. 2023-11-28 03:54:13,956 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-28 03:54:13,956 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-28 03:54:13,956 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:13,956 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-11-28 03:54:13,956 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-11-28 03:54:13,958 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:13,958 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:13,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:13,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:54:13,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:54:13,958 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:13,958 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-28 03:54:13,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:54:13,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:54:13,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:13,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:13,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:54:13,960 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-11-28 03:54:13,960 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@12980ab[Count = 0] remaining members to acquire global barrier 2023-11-28 03:54:13,960 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-11-28 03:54:13,960 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:13,964 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:13,964 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:13,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:13,964 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-11-28 03:54:13,964 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-11-28 03:54:13,964 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,41203,1701143600653' in zk 2023-11-28 03:54:13,964 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:13,965 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-28 03:54:13,966 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-11-28 03:54:13,966 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:54:13,966 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:13,966 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-11-28 03:54:13,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:13,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:54:13,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:54:13,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:54:13,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:54:13,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:13,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:13,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:54:13,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:13,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:13,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,41203,1701143600653': 2023-11-28 03:54:13,968 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-11-28 03:54:13,968 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-11-28 03:54:13,968 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-28 03:54:13,968 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-28 03:54:13,969 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:13,969 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-28 03:54:13,970 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:13,970 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:13,970 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:54:13,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:13,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:54:13,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:54:13,970 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:13,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:13,971 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:54:13,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:54:13,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:54:13,971 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:13,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:13,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:13,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:54:13,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:13,972 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:13,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:13,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:54:13,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:13,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:13,974 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:13,974 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:54:13,974 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:13,974 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:54:13,974 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:54:13,974 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-11-28 03:54:13,975 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-28 03:54:13,974 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:54:13,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:54:13,974 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:13,975 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:13,975 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:13,975 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:13,975 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:13,975 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-11-28 03:54:13,975 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:54:13,975 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-28 03:54:13,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:54:13,980 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(399): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-11-28 03:54:14,075 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-28 03:54:14,077 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-28 03:54:14,080 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-28 03:54:14,082 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-11-28 03:54:14,082 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:14,082 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-11-28 03:54:14,082 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:54:14,082 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-11-28 03:54:14,082 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-11-28 03:54:14,083 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:14,083 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:14,084 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:14,084 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:54:14,084 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:54:14,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:54:14,084 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:14,084 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-28 03:54:14,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:14,085 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:14,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-28 03:54:14,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:14,085 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:14,086 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-11-28 03:54:14,086 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:14,086 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-11-28 03:54:14,086 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:54:14,086 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-11-28 03:54:14,086 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-28 03:54:14,086 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-28 03:54:14,086 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1701143645058.8c6dc3114c8460cdf466d087b23e49ef. 2023-11-28 03:54:14,086 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1701143645058.8c6dc3114c8460cdf466d087b23e49ef. started... 2023-11-28 03:54:14,086 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 8c6dc3114c8460cdf466d087b23e49ef 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-28 03:54:14,499 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015090782fdc5d64eb54a45b8c9b89ed2fd6fe7 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2015090782fdc5d64eb54a45b8c9b89ed2fd6fe7 2023-11-28 03:54:14,905 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyPolicy/8c6dc3114c8460cdf466d087b23e49ef/.tmp/family1/f01464021fde4da2b47d40dac7499ce3 2023-11-28 03:54:14,909 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyPolicy/8c6dc3114c8460cdf466d087b23e49ef/.tmp/family1/f01464021fde4da2b47d40dac7499ce3 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyPolicy/8c6dc3114c8460cdf466d087b23e49ef/family1/f01464021fde4da2b47d40dac7499ce3 2023-11-28 03:54:14,913 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyPolicy/8c6dc3114c8460cdf466d087b23e49ef/family1/f01464021fde4da2b47d40dac7499ce3, entries=1, sequenceid=37, filesize=4.9 K 2023-11-28 03:54:14,914 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 8c6dc3114c8460cdf466d087b23e49ef in 828ms, sequenceid=37, compaction requested=true 2023-11-28 03:54:14,914 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 8c6dc3114c8460cdf466d087b23e49ef: 2023-11-28 03:54:14,914 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1701143645058.8c6dc3114c8460cdf466d087b23e49ef. 2023-11-28 03:54:14,914 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-28 03:54:14,914 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-28 03:54:14,914 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:14,914 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-11-28 03:54:14,914 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-11-28 03:54:14,916 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:14,916 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:14,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:14,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:54:14,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:54:14,917 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:14,917 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-28 03:54:14,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:54:14,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:54:14,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:14,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:14,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:54:14,918 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-11-28 03:54:14,918 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@60295086[Count = 0] remaining members to acquire global barrier 2023-11-28 03:54:14,918 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-11-28 03:54:14,918 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:14,921 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:14,921 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:14,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:14,921 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-11-28 03:54:14,921 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-11-28 03:54:14,921 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,41203,1701143600653' in zk 2023-11-28 03:54:14,921 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:14,921 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-28 03:54:14,922 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:14,922 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-11-28 03:54:14,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:14,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:54:14,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:54:14,923 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:54:14,923 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-11-28 03:54:14,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:54:14,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:54:14,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:14,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:14,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:54:14,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:14,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:14,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,41203,1701143600653': 2023-11-28 03:54:14,925 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-11-28 03:54:14,925 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-11-28 03:54:14,925 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-28 03:54:14,925 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-28 03:54:14,925 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:14,925 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-28 03:54:14,927 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:14,927 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:14,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:14,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:54:14,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:54:14,927 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:14,927 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:54:14,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:14,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:54:14,927 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:54:14,927 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:14,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:54:14,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:14,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:14,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:54:14,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:14,928 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:14,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:14,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:54:14,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:14,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:14,931 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:14,931 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:14,931 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:54:14,931 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:54:14,931 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:14,931 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:54:14,931 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:54:14,931 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(399): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-11-28 03:54:14,931 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:14,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:54:14,931 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-11-28 03:54:14,931 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-28 03:54:14,931 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:14,931 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:14,931 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:14,931 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-11-28 03:54:14,931 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:54:14,932 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-28 03:54:14,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:54:15,032 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-28 03:54:15,033 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-28 03:54:15,037 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-28 03:54:15,038 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-11-28 03:54:15,038 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:15,038 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-11-28 03:54:15,038 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:54:15,039 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-11-28 03:54:15,039 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-11-28 03:54:15,039 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:15,039 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:15,041 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:15,041 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:54:15,042 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:54:15,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:54:15,042 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:15,042 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-28 03:54:15,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:15,042 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:15,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-28 03:54:15,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:15,043 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:15,043 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-11-28 03:54:15,043 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:15,043 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-11-28 03:54:15,043 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:54:15,043 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-11-28 03:54:15,043 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-28 03:54:15,043 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-28 03:54:15,043 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1701143645058.8c6dc3114c8460cdf466d087b23e49ef. 2023-11-28 03:54:15,043 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1701143645058.8c6dc3114c8460cdf466d087b23e49ef. started... 2023-11-28 03:54:15,043 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 8c6dc3114c8460cdf466d087b23e49ef 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-28 03:54:15,457 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151120ac420cb7e46f40daa99665b2ac38b3c5 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151120ac420cb7e46f40daa99665b2ac38b3c5 2023-11-28 03:54:15,862 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyPolicy/8c6dc3114c8460cdf466d087b23e49ef/.tmp/family1/9604acbd0ce04e38bd5f24103fabd3ca 2023-11-28 03:54:15,867 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyPolicy/8c6dc3114c8460cdf466d087b23e49ef/.tmp/family1/9604acbd0ce04e38bd5f24103fabd3ca as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyPolicy/8c6dc3114c8460cdf466d087b23e49ef/family1/9604acbd0ce04e38bd5f24103fabd3ca 2023-11-28 03:54:15,872 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyPolicy/8c6dc3114c8460cdf466d087b23e49ef/family1/9604acbd0ce04e38bd5f24103fabd3ca, entries=1, sequenceid=41, filesize=4.9 K 2023-11-28 03:54:15,873 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 8c6dc3114c8460cdf466d087b23e49ef in 830ms, sequenceid=41, compaction requested=true 2023-11-28 03:54:15,873 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 8c6dc3114c8460cdf466d087b23e49ef: 2023-11-28 03:54:15,873 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1701143645058.8c6dc3114c8460cdf466d087b23e49ef. 2023-11-28 03:54:15,873 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-28 03:54:15,873 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-28 03:54:15,873 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:15,873 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-11-28 03:54:15,873 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-11-28 03:54:15,875 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:15,875 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:15,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:15,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:54:15,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:54:15,875 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:15,875 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-28 03:54:15,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:54:15,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:54:15,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:15,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:15,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:54:15,877 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-11-28 03:54:15,877 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-11-28 03:54:15,877 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@68d1fc11[Count = 0] remaining members to acquire global barrier 2023-11-28 03:54:15,877 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:15,878 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:15,878 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:15,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:15,879 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-11-28 03:54:15,879 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-11-28 03:54:15,879 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,41203,1701143600653' in zk 2023-11-28 03:54:15,879 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:15,879 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-28 03:54:15,880 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-11-28 03:54:15,880 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:15,880 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:54:15,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:15,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:54:15,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:54:15,880 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-11-28 03:54:15,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:54:15,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:54:15,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:15,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:15,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:54:15,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:15,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:15,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,41203,1701143600653': 2023-11-28 03:54:15,883 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-11-28 03:54:15,883 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-11-28 03:54:15,883 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-28 03:54:15,883 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-28 03:54:15,883 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:15,883 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-28 03:54:15,886 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:15,886 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:15,886 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:15,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:15,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:54:15,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:54:15,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:15,886 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:54:15,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:54:15,887 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:54:15,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:54:15,887 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:15,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:15,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:15,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:54:15,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:15,888 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:15,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:15,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:54:15,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:15,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:15,893 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:15,893 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:15,893 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:15,893 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:15,893 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:15,893 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:15,893 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-11-28 03:54:15,893 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-28 03:54:15,893 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:54:15,893 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:15,893 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:54:15,893 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:54:15,893 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:54:15,893 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-11-28 03:54:15,893 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-28 03:54:15,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:54:15,893 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(399): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-11-28 03:54:15,894 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:54:15,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:54:15,993 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-28 03:54:15,994 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-28 03:54:15,996 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=38593] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMajorCompactionWithWeeklyPolicy 2023-11-28 03:54:15,998 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] procedure2.ProcedureExecutor(1028): Stored pid=38, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-11-28 03:54:15,998 DEBUG [PEWorker-2] locking.LockProcedure(309): LOCKED pid=38, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-11-28 03:54:15,999 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyPolicy', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-11-28 03:54:15,999 INFO [PEWorker-2] procedure2.TimeoutExecutorThread(81): ADDED pid=38, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1701144255999 2023-11-28 03:54:16,002 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-11-28 03:54:16,004 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyPolicy' and column 'family1' 2023-11-28 03:54:16,004 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-11-28 03:54:16,006 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] zookeeper.ReadOnlyZKClient(139): Connect 0x0e5b8d27 to 127.0.0.1:55641 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-11-28 03:54:16,013 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] ipc.AbstractRpcClient(189): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@749f2537, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-11-28 03:54:16,013 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-11-28 03:54:16,857 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601035a162857c4af4ee5be00268e4596d61d to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201601035a162857c4af4ee5be00268e4596d61d 2023-11-28 03:54:16,858 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-11-28 03:54:16,871 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(122): Using SIMPLE authentication for service=ClientService, sasl=false 2023-11-28 03:54:16,873 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:41662, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-11-28 03:54:16,880 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(122): Using SIMPLE authentication for service=MasterService, sasl=false 2023-11-28 03:54:16,881 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:41422, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-11-28 03:54:16,889 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/61c196897e9842eb8ad455cf8b3655e0 first=Optional[r3] last=Optional[r6] 2023-11-28 03:54:16,890 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithWeeklyPolicy,,1701143645058.8c6dc3114c8460cdf466d087b23e49ef., hostname=jenkins-hbase4.apache.org,41203,1701143600653, seqNum=2 for row with hfile group [{family1,hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/61c196897e9842eb8ad455cf8b3655e0}] 2023-11-28 03:54:16,894 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HStore(763): Validating hfile at hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/61c196897e9842eb8ad455cf8b3655e0 for inclusion in 8c6dc3114c8460cdf466d087b23e49ef/family1 2023-11-28 03:54:16,896 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HStore(775): HFile bounds: first=r3 last=r6 2023-11-28 03:54:16,896 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HStore(777): Region bounds: first= last= 2023-11-28 03:54:16,896 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HRegion(2446): Flush status journal for 8c6dc3114c8460cdf466d087b23e49ef: 2023-11-28 03:54:16,897 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/61c196897e9842eb8ad455cf8b3655e0 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testMajorCompactionWithWeeklyPolicy__lm4b4t0uofnv8orc8bmgvt2lsges5pb3l458tf1rckuglqlf0la6bg1mmlpe8gs4/family1/61c196897e9842eb8ad455cf8b3655e0 2023-11-28 03:54:16,900 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testMajorCompactionWithWeeklyPolicy__lm4b4t0uofnv8orc8bmgvt2lsges5pb3l458tf1rckuglqlf0la6bg1mmlpe8gs4/family1/61c196897e9842eb8ad455cf8b3655e0 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyPolicy/8c6dc3114c8460cdf466d087b23e49ef/family1/826adefcdfa343cfaa4c805e24f092ec_SeqId_44_ 2023-11-28 03:54:16,900 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HStore(849): Loaded HFile hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testMajorCompactionWithWeeklyPolicy__lm4b4t0uofnv8orc8bmgvt2lsges5pb3l458tf1rckuglqlf0la6bg1mmlpe8gs4/family1/61c196897e9842eb8ad455cf8b3655e0 into 8c6dc3114c8460cdf466d087b23e49ef/family1 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyPolicy/8c6dc3114c8460cdf466d087b23e49ef/family1/826adefcdfa343cfaa4c805e24f092ec_SeqId_44_ - updating store file list. 2023-11-28 03:54:16,904 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HStore(882): Loaded HFile hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyPolicy/8c6dc3114c8460cdf466d087b23e49ef/family1/826adefcdfa343cfaa4c805e24f092ec_SeqId_44_ into 8c6dc3114c8460cdf466d087b23e49ef/family1 2023-11-28 03:54:16,905 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HStore(855): Successfully loaded hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testMajorCompactionWithWeeklyPolicy__lm4b4t0uofnv8orc8bmgvt2lsges5pb3l458tf1rckuglqlf0la6bg1mmlpe8gs4/family1/61c196897e9842eb8ad455cf8b3655e0 into 8c6dc3114c8460cdf466d087b23e49ef/family1 (new location: hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyPolicy/8c6dc3114c8460cdf466d087b23e49ef/family1/826adefcdfa343cfaa4c805e24f092ec_SeqId_44_) 2023-11-28 03:54:16,906 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testMajorCompactionWithWeeklyPolicy__lm4b4t0uofnv8orc8bmgvt2lsges5pb3l458tf1rckuglqlf0la6bg1mmlpe8gs4/family1/61c196897e9842eb8ad455cf8b3655e0 2023-11-28 03:54:16,906 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-11-28 03:54:16,910 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 51214 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-11-28 03:54:16,910 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HStore(1912): 8c6dc3114c8460cdf466d087b23e49ef/family1 is initiating minor compaction 2023-11-28 03:54:16,910 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@369e8493; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-28 03:54:16,910 INFO [RS:0;jenkins-hbase4:41203-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 8c6dc3114c8460cdf466d087b23e49ef/family1 in testMajorCompactionWithWeeklyPolicy,,1701143645058.8c6dc3114c8460cdf466d087b23e49ef. 2023-11-28 03:54:16,910 INFO [RS:0;jenkins-hbase4:41203-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyPolicy/8c6dc3114c8460cdf466d087b23e49ef/family1/967f2a3befe74ada91cd38df5a1f2d39, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyPolicy/8c6dc3114c8460cdf466d087b23e49ef/family1/423d77dca0d54cca89f8eb673fb69407, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyPolicy/8c6dc3114c8460cdf466d087b23e49ef/family1/71b79bd8528445379ad8504e5a0108a4, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyPolicy/8c6dc3114c8460cdf466d087b23e49ef/family1/7b192f29e2a248d49efa66a8fbb93b8a, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyPolicy/8c6dc3114c8460cdf466d087b23e49ef/family1/e9b9e857714947d3a2eb0e3a948b1864, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyPolicy/8c6dc3114c8460cdf466d087b23e49ef/family1/bd8c8840b6f3499291fc5e07644dac99, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyPolicy/8c6dc3114c8460cdf466d087b23e49ef/family1/5017517f4d2846a0bc186136ffa4bd5b, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyPolicy/8c6dc3114c8460cdf466d087b23e49ef/family1/f01464021fde4da2b47d40dac7499ce3, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyPolicy/8c6dc3114c8460cdf466d087b23e49ef/family1/9604acbd0ce04e38bd5f24103fabd3ca, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyPolicy/8c6dc3114c8460cdf466d087b23e49ef/family1/826adefcdfa343cfaa4c805e24f092ec_SeqId_44_] into tmpdir=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyPolicy/8c6dc3114c8460cdf466d087b23e49ef/.tmp, totalSize=50.0 K 2023-11-28 03:54:16,910 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] compactions.Compactor(207): Compacting 967f2a3befe74ada91cd38df5a1f2d39, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-11-28 03:54:16,910 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] compactions.Compactor(207): Compacting 423d77dca0d54cca89f8eb673fb69407, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=13 2023-11-28 03:54:16,910 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] compactions.Compactor(207): Compacting 71b79bd8528445379ad8504e5a0108a4, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=17 2023-11-28 03:54:16,911 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] compactions.Compactor(207): Compacting 7b192f29e2a248d49efa66a8fbb93b8a, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=21 2023-11-28 03:54:16,911 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] compactions.Compactor(207): Compacting e9b9e857714947d3a2eb0e3a948b1864, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=25 2023-11-28 03:54:16,911 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] compactions.Compactor(207): Compacting bd8c8840b6f3499291fc5e07644dac99, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=29 2023-11-28 03:54:16,911 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] compactions.Compactor(207): Compacting 5017517f4d2846a0bc186136ffa4bd5b, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=33 2023-11-28 03:54:16,911 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] compactions.Compactor(207): Compacting f01464021fde4da2b47d40dac7499ce3, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=37 2023-11-28 03:54:16,911 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] compactions.Compactor(207): Compacting 9604acbd0ce04e38bd5f24103fabd3ca, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=41 2023-11-28 03:54:16,911 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] compactions.Compactor(207): Compacting 826adefcdfa343cfaa4c805e24f092ec_SeqId_44_, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=45 2023-11-28 03:54:16,913 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-11-28 03:54:16,915 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-28 03:54:16,917 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151228a32717d3745249039b91112b33480e17 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151228a32717d3745249039b91112b33480e17 2023-11-28 03:54:16,918 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151231e5b27507111c445e87d3eb16d0bba565 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151231e5b27507111c445e87d3eb16d0bba565 2023-11-28 03:54:16,920 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20160101845a31f1b34444f993da6b8d4b7475a8 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20160101845a31f1b34444f993da6b8d4b7475a8 2023-11-28 03:54:16,921 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2016010322855a10b7164ee899d0737e16f47433 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2016010322855a10b7164ee899d0737e16f47433 2023-11-28 03:54:16,921 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyPolicy 4 expired mob files are deleted 2023-11-28 03:54:16,921 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228' 2023-11-28 03:54:16,922 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-11-28 03:54:16,922 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-11-28 03:54:16,922 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-11-28 03:54:16,922 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-11-28 03:54:16,922 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-11-28 03:54:16,922 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-11-28 03:54:16,922 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-11-28 03:54:16,922 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-11-28 03:54:16,953 INFO [RS:0;jenkins-hbase4:41203-shortCompactions-0] throttle.PressureAwareThroughputController(145): 8c6dc3114c8460cdf466d087b23e49ef#family1#compaction#48 average throughput is 1.36 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-11-28 03:54:17,374 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyPolicy/8c6dc3114c8460cdf466d087b23e49ef/.tmp/family1/57c3537cde4240058136b83ccf1654d2 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyPolicy/8c6dc3114c8460cdf466d087b23e49ef/family1/57c3537cde4240058136b83ccf1654d2 2023-11-28 03:54:17,380 INFO [RS:0;jenkins-hbase4:41203-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in 8c6dc3114c8460cdf466d087b23e49ef/family1 of 8c6dc3114c8460cdf466d087b23e49ef into 57c3537cde4240058136b83ccf1654d2(size=6.6 K), total size for store is 171.6 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-11-28 03:54:17,381 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 8c6dc3114c8460cdf466d087b23e49ef: 2023-11-28 03:54:17,383 INFO [RS:0;jenkins-hbase4:41203-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyPolicy,,1701143645058.8c6dc3114c8460cdf466d087b23e49ef., storeName=family1, fileCount=10, fileSize=50.0 K (5.0 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 5.5 K), priority=2, time=1701143656910; duration=0sec 2023-11-28 03:54:17,384 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-28 03:54:17,394 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyPolicy/8c6dc3114c8460cdf466d087b23e49ef/family1/967f2a3befe74ada91cd38df5a1f2d39, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyPolicy/8c6dc3114c8460cdf466d087b23e49ef/family1/423d77dca0d54cca89f8eb673fb69407, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyPolicy/8c6dc3114c8460cdf466d087b23e49ef/family1/71b79bd8528445379ad8504e5a0108a4, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyPolicy/8c6dc3114c8460cdf466d087b23e49ef/family1/7b192f29e2a248d49efa66a8fbb93b8a, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyPolicy/8c6dc3114c8460cdf466d087b23e49ef/family1/e9b9e857714947d3a2eb0e3a948b1864, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyPolicy/8c6dc3114c8460cdf466d087b23e49ef/family1/bd8c8840b6f3499291fc5e07644dac99, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyPolicy/8c6dc3114c8460cdf466d087b23e49ef/family1/5017517f4d2846a0bc186136ffa4bd5b, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyPolicy/8c6dc3114c8460cdf466d087b23e49ef/family1/f01464021fde4da2b47d40dac7499ce3, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyPolicy/8c6dc3114c8460cdf466d087b23e49ef/family1/9604acbd0ce04e38bd5f24103fabd3ca, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyPolicy/8c6dc3114c8460cdf466d087b23e49ef/family1/826adefcdfa343cfaa4c805e24f092ec_SeqId_44_] to archive 2023-11-28 03:54:17,395 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-28 03:54:17,397 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyPolicy/8c6dc3114c8460cdf466d087b23e49ef/family1/967f2a3befe74ada91cd38df5a1f2d39 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionWithWeeklyPolicy/8c6dc3114c8460cdf466d087b23e49ef/family1/967f2a3befe74ada91cd38df5a1f2d39 2023-11-28 03:54:17,398 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyPolicy/8c6dc3114c8460cdf466d087b23e49ef/family1/423d77dca0d54cca89f8eb673fb69407 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionWithWeeklyPolicy/8c6dc3114c8460cdf466d087b23e49ef/family1/423d77dca0d54cca89f8eb673fb69407 2023-11-28 03:54:17,399 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyPolicy/8c6dc3114c8460cdf466d087b23e49ef/family1/71b79bd8528445379ad8504e5a0108a4 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionWithWeeklyPolicy/8c6dc3114c8460cdf466d087b23e49ef/family1/71b79bd8528445379ad8504e5a0108a4 2023-11-28 03:54:17,400 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyPolicy/8c6dc3114c8460cdf466d087b23e49ef/family1/7b192f29e2a248d49efa66a8fbb93b8a to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionWithWeeklyPolicy/8c6dc3114c8460cdf466d087b23e49ef/family1/7b192f29e2a248d49efa66a8fbb93b8a 2023-11-28 03:54:17,401 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyPolicy/8c6dc3114c8460cdf466d087b23e49ef/family1/e9b9e857714947d3a2eb0e3a948b1864 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionWithWeeklyPolicy/8c6dc3114c8460cdf466d087b23e49ef/family1/e9b9e857714947d3a2eb0e3a948b1864 2023-11-28 03:54:17,402 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyPolicy/8c6dc3114c8460cdf466d087b23e49ef/family1/bd8c8840b6f3499291fc5e07644dac99 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionWithWeeklyPolicy/8c6dc3114c8460cdf466d087b23e49ef/family1/bd8c8840b6f3499291fc5e07644dac99 2023-11-28 03:54:17,403 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyPolicy/8c6dc3114c8460cdf466d087b23e49ef/family1/5017517f4d2846a0bc186136ffa4bd5b to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionWithWeeklyPolicy/8c6dc3114c8460cdf466d087b23e49ef/family1/5017517f4d2846a0bc186136ffa4bd5b 2023-11-28 03:54:17,404 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyPolicy/8c6dc3114c8460cdf466d087b23e49ef/family1/f01464021fde4da2b47d40dac7499ce3 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionWithWeeklyPolicy/8c6dc3114c8460cdf466d087b23e49ef/family1/f01464021fde4da2b47d40dac7499ce3 2023-11-28 03:54:17,405 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyPolicy/8c6dc3114c8460cdf466d087b23e49ef/family1/9604acbd0ce04e38bd5f24103fabd3ca to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionWithWeeklyPolicy/8c6dc3114c8460cdf466d087b23e49ef/family1/9604acbd0ce04e38bd5f24103fabd3ca 2023-11-28 03:54:17,406 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyPolicy/8c6dc3114c8460cdf466d087b23e49ef/family1/826adefcdfa343cfaa4c805e24f092ec_SeqId_44_ to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionWithWeeklyPolicy/8c6dc3114c8460cdf466d087b23e49ef/family1/826adefcdfa343cfaa4c805e24f092ec_SeqId_44_ 2023-11-28 03:54:17,799 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512051f6d39df6a444e24a3d37161412ae406 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201512051f6d39df6a444e24a3d37161412ae406 2023-11-28 03:54:17,800 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-11-28 03:54:17,827 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/e758a0a7689840b8a9b75bc684c72642 first=Optional[r00] last=Optional[r2] 2023-11-28 03:54:17,827 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithWeeklyPolicy,,1701143645058.8c6dc3114c8460cdf466d087b23e49ef., hostname=jenkins-hbase4.apache.org,41203,1701143600653, seqNum=2 for row with hfile group [{family1,hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/e758a0a7689840b8a9b75bc684c72642}] 2023-11-28 03:54:17,829 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HStore(763): Validating hfile at hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/e758a0a7689840b8a9b75bc684c72642 for inclusion in 8c6dc3114c8460cdf466d087b23e49ef/family1 2023-11-28 03:54:17,838 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HStore(775): HFile bounds: first=r00 last=r2 2023-11-28 03:54:17,839 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HStore(777): Region bounds: first= last= 2023-11-28 03:54:17,839 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HRegion(2446): Flush status journal for 8c6dc3114c8460cdf466d087b23e49ef: 2023-11-28 03:54:17,840 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/e758a0a7689840b8a9b75bc684c72642 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testMajorCompactionWithWeeklyPolicy__h9t2mj9o4fodsgknimlfljj0pavupc6ah77pv69h2qq6ocs1c3dcie9q9c3npab2/family1/e758a0a7689840b8a9b75bc684c72642 2023-11-28 03:54:17,842 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testMajorCompactionWithWeeklyPolicy__h9t2mj9o4fodsgknimlfljj0pavupc6ah77pv69h2qq6ocs1c3dcie9q9c3npab2/family1/e758a0a7689840b8a9b75bc684c72642 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyPolicy/8c6dc3114c8460cdf466d087b23e49ef/family1/5c3bd7cd5ecf494eacdd6559c0931ab5_SeqId_47_ 2023-11-28 03:54:17,843 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HStore(849): Loaded HFile hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testMajorCompactionWithWeeklyPolicy__h9t2mj9o4fodsgknimlfljj0pavupc6ah77pv69h2qq6ocs1c3dcie9q9c3npab2/family1/e758a0a7689840b8a9b75bc684c72642 into 8c6dc3114c8460cdf466d087b23e49ef/family1 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyPolicy/8c6dc3114c8460cdf466d087b23e49ef/family1/5c3bd7cd5ecf494eacdd6559c0931ab5_SeqId_47_ - updating store file list. 2023-11-28 03:54:17,847 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HStore(882): Loaded HFile hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyPolicy/8c6dc3114c8460cdf466d087b23e49ef/family1/5c3bd7cd5ecf494eacdd6559c0931ab5_SeqId_47_ into 8c6dc3114c8460cdf466d087b23e49ef/family1 2023-11-28 03:54:17,847 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HStore(855): Successfully loaded hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testMajorCompactionWithWeeklyPolicy__h9t2mj9o4fodsgknimlfljj0pavupc6ah77pv69h2qq6ocs1c3dcie9q9c3npab2/family1/e758a0a7689840b8a9b75bc684c72642 into 8c6dc3114c8460cdf466d087b23e49ef/family1 (new location: hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyPolicy/8c6dc3114c8460cdf466d087b23e49ef/family1/5c3bd7cd5ecf494eacdd6559c0931ab5_SeqId_47_) 2023-11-28 03:54:17,848 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testMajorCompactionWithWeeklyPolicy__h9t2mj9o4fodsgknimlfljj0pavupc6ah77pv69h2qq6ocs1c3dcie9q9c3npab2/family1/e758a0a7689840b8a9b75bc684c72642 2023-11-28 03:54:17,848 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-11-28 03:54:17,849 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 345120 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-11-28 03:54:17,849 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HStore(1912): 8c6dc3114c8460cdf466d087b23e49ef/family1 is initiating minor compaction (all files) 2023-11-28 03:54:17,849 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@28a4fe50; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-28 03:54:17,849 INFO [RS:0;jenkins-hbase4:41203-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 8c6dc3114c8460cdf466d087b23e49ef/family1 in testMajorCompactionWithWeeklyPolicy,,1701143645058.8c6dc3114c8460cdf466d087b23e49ef. 2023-11-28 03:54:17,849 INFO [RS:0;jenkins-hbase4:41203-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyPolicy/8c6dc3114c8460cdf466d087b23e49ef/family1/299aee69722249a898d53379abc8c8c3, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyPolicy/8c6dc3114c8460cdf466d087b23e49ef/family1/57c3537cde4240058136b83ccf1654d2, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyPolicy/8c6dc3114c8460cdf466d087b23e49ef/family1/5c3bd7cd5ecf494eacdd6559c0931ab5_SeqId_47_] into tmpdir=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyPolicy/8c6dc3114c8460cdf466d087b23e49ef/.tmp, totalSize=337.0 K 2023-11-28 03:54:17,850 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] compactions.Compactor(207): Compacting 299aee69722249a898d53379abc8c8c3, keycount=1000, bloomtype=ROW, size=165.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878824053 2023-11-28 03:54:17,850 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] compactions.Compactor(207): Compacting 57c3537cde4240058136b83ccf1654d2, keycount=9, bloomtype=ROW, size=6.6 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621224053 2023-11-28 03:54:17,851 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] compactions.Compactor(207): Compacting 5c3bd7cd5ecf494eacdd6559c0931ab5_SeqId_47_, keycount=1002, bloomtype=ROW, size=165.4 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448878824053 2023-11-28 03:54:17,851 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-11-28 03:54:17,852 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-28 03:54:17,853 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2015113004e9084677c1492894955371e22862a6 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2015113004e9084677c1492894955371e22862a6 2023-11-28 03:54:17,855 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151201f5b083ae53594740b7bd7b5806bf03ee to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151201f5b083ae53594740b7bd7b5806bf03ee 2023-11-28 03:54:17,856 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151205833d85e416cd4a018dd0d85b4759ad73 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151205833d85e416cd4a018dd0d85b4759ad73 2023-11-28 03:54:17,856 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyPolicy 3 expired mob files are deleted 2023-11-28 03:54:17,856 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130' 2023-11-28 03:54:17,857 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-11-28 03:54:17,857 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] compactions.PartitionedMobCompactor(354): After compaction, there are 5 mob files. table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-11-28 03:54:17,857 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyPolicy', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-11-28 03:54:17,858 DEBUG [PEWorker-5] locking.LockProcedure(242): UNLOCKED pid=38, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-11-28 03:54:17,859 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=38, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE in 1.8620 sec 2023-11-28 03:54:17,869 INFO [RS:0;jenkins-hbase4:41203-shortCompactions-0] throttle.PressureAwareThroughputController(145): 8c6dc3114c8460cdf466d087b23e49ef#family1#compaction#49 average throughput is 15.46 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-11-28 03:54:17,979 INFO [Listener at localhost/42853] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyPolicy Thread=261 (was 250) - Thread LEAK? -, OpenFileDescriptor=648 (was 613) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=196 (was 214), ProcessCount=172 (was 172), AvailableMemoryMB=6978 (was 6983) 2023-11-28 03:54:17,993 INFO [Listener at localhost/42853] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly Thread=261, OpenFileDescriptor=648, MaxFileDescriptor=60000, SystemLoadAverage=196, ProcessCount=172, AvailableMemoryMB=6977 2023-11-28 03:54:17,995 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-11-28 03:54:17,996 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] procedure2.ProcedureExecutor(1028): Stored pid=39, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:17,997 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_PRE_OPERATION 2023-11-28 03:54:17,997 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(700): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly" procId is: 39 2023-11-28 03:54:17,998 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-11-28 03:54:17,998 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(1230): Checking to see if procedure is done pid=39 2023-11-28 03:54:17,999 DEBUG [HFileArchiver-9] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a075e19a5494ecfa5418f707da74e5f0 2023-11-28 03:54:18,000 DEBUG [HFileArchiver-9] backup.HFileArchiver(153): Directory hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a075e19a5494ecfa5418f707da74e5f0 empty. 2023-11-28 03:54:18,000 DEBUG [HFileArchiver-9] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a075e19a5494ecfa5418f707da74e5f0 2023-11-28 03:54:18,000 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(328): Archived testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly regions 2023-11-28 03:54:18,099 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(1230): Checking to see if procedure is done pid=39 2023-11-28 03:54:18,300 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(1230): Checking to see if procedure is done pid=39 2023-11-28 03:54:18,408 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000001 2023-11-28 03:54:18,409 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(7675): creating {ENCODED => a075e19a5494ecfa5418f707da74e5f0, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701143657995.a075e19a5494ecfa5418f707da74e5f0.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}, regionDir=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/.tmp 2023-11-28 03:54:18,601 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(1230): Checking to see if procedure is done pid=39 2023-11-28 03:54:18,815 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701143657995.a075e19a5494ecfa5418f707da74e5f0.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-28 03:54:18,815 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1604): Closing a075e19a5494ecfa5418f707da74e5f0, disabling compactions & flushes 2023-11-28 03:54:18,816 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701143657995.a075e19a5494ecfa5418f707da74e5f0. 2023-11-28 03:54:18,816 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701143657995.a075e19a5494ecfa5418f707da74e5f0. 2023-11-28 03:54:18,816 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701143657995.a075e19a5494ecfa5418f707da74e5f0. after waiting 0 ms 2023-11-28 03:54:18,816 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701143657995.a075e19a5494ecfa5418f707da74e5f0. 2023-11-28 03:54:18,816 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701143657995.a075e19a5494ecfa5418f707da74e5f0. 2023-11-28 03:54:18,816 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1558): Region close journal for a075e19a5494ecfa5418f707da74e5f0: 2023-11-28 03:54:18,819 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_ADD_TO_META 2023-11-28 03:54:18,820 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701143657995.a075e19a5494ecfa5418f707da74e5f0.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1701143658820"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1701143658820"}]},"ts":"1701143658820"} 2023-11-28 03:54:18,822 INFO [PEWorker-4] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-11-28 03:54:18,822 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-11-28 03:54:18,823 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1701143658823"}]},"ts":"1701143658823"} 2023-11-28 03:54:18,824 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, state=ENABLING in hbase:meta 2023-11-28 03:54:18,828 INFO [PEWorker-4] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=40, ppid=39, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=a075e19a5494ecfa5418f707da74e5f0, ASSIGN}] 2023-11-28 03:54:18,830 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=40, ppid=39, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=a075e19a5494ecfa5418f707da74e5f0, ASSIGN 2023-11-28 03:54:18,830 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=40, ppid=39, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=a075e19a5494ecfa5418f707da74e5f0, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,41203,1701143600653; forceNewPlan=false, retain=false 2023-11-28 03:54:18,981 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=40 updating hbase:meta row=a075e19a5494ecfa5418f707da74e5f0, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:18,982 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701143657995.a075e19a5494ecfa5418f707da74e5f0.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1701143658981"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1701143658981"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1701143658981"}]},"ts":"1701143658981"} 2023-11-28 03:54:18,984 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=41, ppid=40, state=RUNNABLE; OpenRegionProcedure a075e19a5494ecfa5418f707da74e5f0, server=jenkins-hbase4.apache.org,41203,1701143600653}] 2023-11-28 03:54:19,091 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyPolicy/8c6dc3114c8460cdf466d087b23e49ef/.tmp/family1/a74fc813d52a48b3b5713dea5573318d as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyPolicy/8c6dc3114c8460cdf466d087b23e49ef/family1/a74fc813d52a48b3b5713dea5573318d 2023-11-28 03:54:19,096 INFO [RS:0;jenkins-hbase4:41203-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 8c6dc3114c8460cdf466d087b23e49ef/family1 of 8c6dc3114c8460cdf466d087b23e49ef into a74fc813d52a48b3b5713dea5573318d(size=166.4 K), total size for store is 166.4 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-11-28 03:54:19,096 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 8c6dc3114c8460cdf466d087b23e49ef: 2023-11-28 03:54:19,096 INFO [RS:0;jenkins-hbase4:41203-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyPolicy,,1701143645058.8c6dc3114c8460cdf466d087b23e49ef., storeName=family1, fileCount=3, fileSize=337.0 K (165.1 K, 6.6 K, 165.4 K), priority=2, time=1701143657849; duration=1sec 2023-11-28 03:54:19,097 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-28 03:54:19,103 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(1230): Checking to see if procedure is done pid=39 2023-11-28 03:54:19,135 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:19,138 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701143657995.a075e19a5494ecfa5418f707da74e5f0. 2023-11-28 03:54:19,138 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => a075e19a5494ecfa5418f707da74e5f0, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701143657995.a075e19a5494ecfa5418f707da74e5f0.', STARTKEY => '', ENDKEY => ''} 2023-11-28 03:54:19,139 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly a075e19a5494ecfa5418f707da74e5f0 2023-11-28 03:54:19,139 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701143657995.a075e19a5494ecfa5418f707da74e5f0.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-28 03:54:19,139 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for a075e19a5494ecfa5418f707da74e5f0 2023-11-28 03:54:19,139 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for a075e19a5494ecfa5418f707da74e5f0 2023-11-28 03:54:19,140 INFO [StoreOpener-a075e19a5494ecfa5418f707da74e5f0-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region a075e19a5494ecfa5418f707da74e5f0 2023-11-28 03:54:19,142 DEBUG [StoreOpener-a075e19a5494ecfa5418f707da74e5f0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a075e19a5494ecfa5418f707da74e5f0/family1 2023-11-28 03:54:19,142 DEBUG [StoreOpener-a075e19a5494ecfa5418f707da74e5f0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a075e19a5494ecfa5418f707da74e5f0/family1 2023-11-28 03:54:19,142 INFO [StoreOpener-a075e19a5494ecfa5418f707da74e5f0-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a075e19a5494ecfa5418f707da74e5f0 columnFamilyName family1 2023-11-28 03:54:19,143 INFO [StoreOpener-a075e19a5494ecfa5418f707da74e5f0-1] regionserver.HStore(310): Store=a075e19a5494ecfa5418f707da74e5f0/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-28 03:54:19,144 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a075e19a5494ecfa5418f707da74e5f0 2023-11-28 03:54:19,145 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a075e19a5494ecfa5418f707da74e5f0 2023-11-28 03:54:19,147 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for a075e19a5494ecfa5418f707da74e5f0 2023-11-28 03:54:19,148 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a075e19a5494ecfa5418f707da74e5f0/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-11-28 03:54:19,149 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened a075e19a5494ecfa5418f707da74e5f0; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=62167897, jitterRate=-0.07362614572048187}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-11-28 03:54:19,149 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for a075e19a5494ecfa5418f707da74e5f0: 2023-11-28 03:54:19,149 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2339): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701143657995.a075e19a5494ecfa5418f707da74e5f0., pid=41, masterSystemTime=1701143659135 2023-11-28 03:54:19,151 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2366): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701143657995.a075e19a5494ecfa5418f707da74e5f0. 2023-11-28 03:54:19,151 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701143657995.a075e19a5494ecfa5418f707da74e5f0. 2023-11-28 03:54:19,151 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=40 updating hbase:meta row=a075e19a5494ecfa5418f707da74e5f0, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:19,152 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701143657995.a075e19a5494ecfa5418f707da74e5f0.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1701143659151"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1701143659151"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1701143659151"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1701143659151"}]},"ts":"1701143659151"} 2023-11-28 03:54:19,154 INFO [PEWorker-5] procedure2.ProcedureExecutor(1825): Finished subprocedure pid=41, resume processing ppid=40 2023-11-28 03:54:19,154 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=41, ppid=40, state=SUCCESS; OpenRegionProcedure a075e19a5494ecfa5418f707da74e5f0, server=jenkins-hbase4.apache.org,41203,1701143600653 in 170 msec 2023-11-28 03:54:19,156 INFO [PEWorker-4] procedure2.ProcedureExecutor(1825): Finished subprocedure pid=40, resume processing ppid=39 2023-11-28 03:54:19,156 INFO [PEWorker-4] procedure2.ProcedureExecutor(1409): Finished pid=40, ppid=39, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=a075e19a5494ecfa5418f707da74e5f0, ASSIGN in 326 msec 2023-11-28 03:54:19,157 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-11-28 03:54:19,157 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1701143659157"}]},"ts":"1701143659157"} 2023-11-28 03:54:19,158 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, state=ENABLED in hbase:meta 2023-11-28 03:54:19,161 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_POST_OPERATION 2023-11-28 03:54:19,162 INFO [PEWorker-3] procedure2.ProcedureExecutor(1409): Finished pid=39, state=SUCCESS; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly in 1.1660 sec 2023-11-28 03:54:19,184 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyPolicy/8c6dc3114c8460cdf466d087b23e49ef/family1/299aee69722249a898d53379abc8c8c3, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyPolicy/8c6dc3114c8460cdf466d087b23e49ef/family1/57c3537cde4240058136b83ccf1654d2, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyPolicy/8c6dc3114c8460cdf466d087b23e49ef/family1/5c3bd7cd5ecf494eacdd6559c0931ab5_SeqId_47_] to archive 2023-11-28 03:54:19,185 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-28 03:54:19,186 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyPolicy/8c6dc3114c8460cdf466d087b23e49ef/family1/299aee69722249a898d53379abc8c8c3 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionWithWeeklyPolicy/8c6dc3114c8460cdf466d087b23e49ef/family1/299aee69722249a898d53379abc8c8c3 2023-11-28 03:54:19,188 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyPolicy/8c6dc3114c8460cdf466d087b23e49ef/family1/57c3537cde4240058136b83ccf1654d2 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionWithWeeklyPolicy/8c6dc3114c8460cdf466d087b23e49ef/family1/57c3537cde4240058136b83ccf1654d2 2023-11-28 03:54:19,189 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyPolicy/8c6dc3114c8460cdf466d087b23e49ef/family1/5c3bd7cd5ecf494eacdd6559c0931ab5_SeqId_47_ to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionWithWeeklyPolicy/8c6dc3114c8460cdf466d087b23e49ef/family1/5c3bd7cd5ecf494eacdd6559c0931ab5_SeqId_47_ 2023-11-28 03:54:20,104 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(1230): Checking to see if procedure is done pid=39 2023-11-28 03:54:20,104 INFO [Listener at localhost/42853] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, procId: 39 completed 2023-11-28 03:54:20,131 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-28 03:54:20,133 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:20,134 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-11-28 03:54:20,134 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:54:20,134 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-11-28 03:54:20,134 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-11-28 03:54:20,134 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:20,134 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:20,136 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:54:20,136 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:20,136 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:54:20,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:54:20,136 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:20,136 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-28 03:54:20,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:20,137 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:20,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-28 03:54:20,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:20,137 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:20,137 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:20,137 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-11-28 03:54:20,137 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:54:20,137 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-11-28 03:54:20,138 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-28 03:54:20,138 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-28 03:54:20,138 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701143657995.a075e19a5494ecfa5418f707da74e5f0. 2023-11-28 03:54:20,138 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701143657995.a075e19a5494ecfa5418f707da74e5f0. started... 2023-11-28 03:54:20,138 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing a075e19a5494ecfa5418f707da74e5f0 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-11-28 03:54:20,560 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015113084aeaab323b645c7834083383ceec7d6 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015113084aeaab323b645c7834083383ceec7d6 2023-11-28 03:54:20,964 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a075e19a5494ecfa5418f707da74e5f0/.tmp/family1/633e489f34d44208a9e77c96ae35ea69 2023-11-28 03:54:20,969 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a075e19a5494ecfa5418f707da74e5f0/.tmp/family1/633e489f34d44208a9e77c96ae35ea69 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a075e19a5494ecfa5418f707da74e5f0/family1/633e489f34d44208a9e77c96ae35ea69 2023-11-28 03:54:20,973 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a075e19a5494ecfa5418f707da74e5f0/family1/633e489f34d44208a9e77c96ae35ea69, entries=1000, sequenceid=5, filesize=191.4 K 2023-11-28 03:54:20,974 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for a075e19a5494ecfa5418f707da74e5f0 in 835ms, sequenceid=5, compaction requested=false 2023-11-28 03:54:20,974 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-11-28 03:54:20,974 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for a075e19a5494ecfa5418f707da74e5f0: 2023-11-28 03:54:20,974 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701143657995.a075e19a5494ecfa5418f707da74e5f0. 2023-11-28 03:54:20,974 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-28 03:54:20,974 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-28 03:54:20,974 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:20,974 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-11-28 03:54:20,974 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-11-28 03:54:20,976 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:20,976 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:20,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:20,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:54:20,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:54:20,977 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:20,977 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-28 03:54:20,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:54:20,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:54:20,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:20,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:20,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:54:20,978 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-11-28 03:54:20,978 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@17f271ce[Count = 0] remaining members to acquire global barrier 2023-11-28 03:54:20,978 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-11-28 03:54:20,978 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:20,981 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:20,981 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:20,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:20,981 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-11-28 03:54:20,981 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-11-28 03:54:20,981 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,41203,1701143600653' in zk 2023-11-28 03:54:20,981 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:20,981 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-28 03:54:20,983 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:20,983 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-11-28 03:54:20,983 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:54:20,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:20,983 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-11-28 03:54:20,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:54:20,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:54:20,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:54:20,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:54:20,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:20,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:20,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:54:20,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:20,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:20,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,41203,1701143600653': 2023-11-28 03:54:20,985 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-11-28 03:54:20,985 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-11-28 03:54:20,985 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-28 03:54:20,985 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-28 03:54:20,986 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:20,986 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-28 03:54:20,987 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:20,987 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:20,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:20,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:54:20,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:54:20,987 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:54:20,987 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:20,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:20,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:54:20,988 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:20,988 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:54:20,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:54:20,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:20,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:20,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:54:20,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:20,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:20,989 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:20,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:54:20,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:20,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:20,991 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:20,991 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:20,992 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:54:20,992 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:20,992 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:54:20,992 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:54:20,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:54:20,992 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-11-28 03:54:20,992 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-28 03:54:20,992 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:20,992 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:54:20,992 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:20,992 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:54:20,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:54:20,992 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-11-28 03:54:20,992 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:20,992 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-28 03:54:20,992 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:21,092 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-28 03:54:21,094 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-28 03:54:21,098 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-28 03:54:21,099 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-11-28 03:54:21,099 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:21,099 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-11-28 03:54:21,099 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:54:21,100 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-11-28 03:54:21,100 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-11-28 03:54:21,100 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:21,100 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:21,103 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:54:21,103 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:21,103 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:54:21,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:54:21,103 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:21,103 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-28 03:54:21,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:21,104 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:21,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-28 03:54:21,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:21,104 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:21,104 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-11-28 03:54:21,104 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:21,104 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-11-28 03:54:21,104 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:54:21,104 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-11-28 03:54:21,104 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-28 03:54:21,105 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-28 03:54:21,105 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701143657995.a075e19a5494ecfa5418f707da74e5f0. 2023-11-28 03:54:21,105 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701143657995.a075e19a5494ecfa5418f707da74e5f0. started... 2023-11-28 03:54:21,105 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing a075e19a5494ecfa5418f707da74e5f0 1/1 column families, dataSize=74 B heapSize=408 B 2023-11-28 03:54:21,524 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015112884d93886d19c445b8c9d84378b7f7802 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015112884d93886d19c445b8c9d84378b7f7802 2023-11-28 03:54:21,929 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a075e19a5494ecfa5418f707da74e5f0/.tmp/family1/b882c18da385433b94ad633b72e7a7c5 2023-11-28 03:54:21,935 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a075e19a5494ecfa5418f707da74e5f0/.tmp/family1/b882c18da385433b94ad633b72e7a7c5 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a075e19a5494ecfa5418f707da74e5f0/family1/b882c18da385433b94ad633b72e7a7c5 2023-11-28 03:54:21,940 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a075e19a5494ecfa5418f707da74e5f0/family1/b882c18da385433b94ad633b72e7a7c5, entries=1, sequenceid=9, filesize=5.0 K 2023-11-28 03:54:21,941 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for a075e19a5494ecfa5418f707da74e5f0 in 836ms, sequenceid=9, compaction requested=false 2023-11-28 03:54:21,941 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for a075e19a5494ecfa5418f707da74e5f0: 2023-11-28 03:54:21,941 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701143657995.a075e19a5494ecfa5418f707da74e5f0. 2023-11-28 03:54:21,941 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-28 03:54:21,941 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-28 03:54:21,941 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:21,941 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-11-28 03:54:21,941 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-11-28 03:54:21,943 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:21,943 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:21,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:21,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:54:21,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:54:21,943 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:21,943 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-28 03:54:21,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:54:21,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:54:21,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:21,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:21,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:54:21,945 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-11-28 03:54:21,945 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5a939743[Count = 0] remaining members to acquire global barrier 2023-11-28 03:54:21,945 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-11-28 03:54:21,945 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:21,946 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:21,947 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:21,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:21,947 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-11-28 03:54:21,947 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-11-28 03:54:21,947 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,41203,1701143600653' in zk 2023-11-28 03:54:21,947 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:21,947 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-28 03:54:21,954 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:21,954 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-11-28 03:54:21,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:21,954 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:54:21,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:54:21,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:54:21,954 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-11-28 03:54:21,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:54:21,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:54:21,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:21,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:21,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:54:21,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:21,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:21,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,41203,1701143600653': 2023-11-28 03:54:21,957 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-11-28 03:54:21,957 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-11-28 03:54:21,957 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-28 03:54:21,957 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-28 03:54:21,957 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:21,957 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-28 03:54:21,959 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:21,959 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:21,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:21,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:54:21,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:54:21,959 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:21,959 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:54:21,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:21,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:54:21,959 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:21,959 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:54:21,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:54:21,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:21,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:21,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:54:21,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:21,961 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:21,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:21,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:54:21,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:21,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:21,966 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:21,966 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:21,966 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:54:21,966 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:21,966 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:54:21,966 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:21,966 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:54:21,966 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:54:21,966 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(399): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-11-28 03:54:21,966 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-11-28 03:54:21,966 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-28 03:54:21,966 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:21,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:54:21,966 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:21,966 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:21,966 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-11-28 03:54:21,966 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-28 03:54:21,966 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:54:21,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:54:22,067 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-28 03:54:22,068 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-28 03:54:22,072 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-28 03:54:22,073 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-11-28 03:54:22,073 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:22,074 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-11-28 03:54:22,074 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:54:22,074 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-11-28 03:54:22,074 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-11-28 03:54:22,074 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:22,074 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:22,076 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:22,076 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:54:22,076 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:54:22,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:54:22,077 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:22,077 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-28 03:54:22,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:22,077 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:22,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-28 03:54:22,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:22,078 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:22,078 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-11-28 03:54:22,078 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:22,078 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-11-28 03:54:22,078 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:54:22,078 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-11-28 03:54:22,078 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-28 03:54:22,078 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-28 03:54:22,078 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701143657995.a075e19a5494ecfa5418f707da74e5f0. 2023-11-28 03:54:22,078 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701143657995.a075e19a5494ecfa5418f707da74e5f0. started... 2023-11-28 03:54:22,078 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing a075e19a5494ecfa5418f707da74e5f0 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-28 03:54:22,493 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512015d75b11b3f36416eb8f97fa9f88a6984 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201512015d75b11b3f36416eb8f97fa9f88a6984 2023-11-28 03:54:22,899 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a075e19a5494ecfa5418f707da74e5f0/.tmp/family1/61ed4c90dc4d472d96eef3d76134d373 2023-11-28 03:54:22,904 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a075e19a5494ecfa5418f707da74e5f0/.tmp/family1/61ed4c90dc4d472d96eef3d76134d373 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a075e19a5494ecfa5418f707da74e5f0/family1/61ed4c90dc4d472d96eef3d76134d373 2023-11-28 03:54:22,908 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a075e19a5494ecfa5418f707da74e5f0/family1/61ed4c90dc4d472d96eef3d76134d373, entries=1, sequenceid=13, filesize=5.0 K 2023-11-28 03:54:22,909 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for a075e19a5494ecfa5418f707da74e5f0 in 831ms, sequenceid=13, compaction requested=true 2023-11-28 03:54:22,909 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for a075e19a5494ecfa5418f707da74e5f0: 2023-11-28 03:54:22,909 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701143657995.a075e19a5494ecfa5418f707da74e5f0. 2023-11-28 03:54:22,909 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-28 03:54:22,910 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-28 03:54:22,910 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:22,910 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-11-28 03:54:22,910 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-11-28 03:54:22,912 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:22,912 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:22,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:22,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:54:22,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:54:22,912 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:22,912 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-28 03:54:22,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:54:22,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:54:22,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:22,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:22,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:54:22,913 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-11-28 03:54:22,914 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@709441f7[Count = 0] remaining members to acquire global barrier 2023-11-28 03:54:22,914 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-11-28 03:54:22,914 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:22,915 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:22,915 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:22,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:22,915 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-11-28 03:54:22,915 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-11-28 03:54:22,915 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,41203,1701143600653' in zk 2023-11-28 03:54:22,915 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:22,915 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-28 03:54:22,918 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:22,918 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-11-28 03:54:22,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:22,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:54:22,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:54:22,918 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:54:22,918 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-11-28 03:54:22,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:54:22,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:54:22,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:22,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:22,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:54:22,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:22,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:22,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,41203,1701143600653': 2023-11-28 03:54:22,920 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-11-28 03:54:22,920 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-11-28 03:54:22,920 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-28 03:54:22,920 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-28 03:54:22,920 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:22,920 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-28 03:54:22,922 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:22,922 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:22,922 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:22,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:22,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:22,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:54:22,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:54:22,922 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:54:22,922 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:54:22,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:54:22,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:54:22,923 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:22,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:22,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:22,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:54:22,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:22,924 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:22,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:22,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:54:22,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:22,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:22,926 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:22,926 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:22,926 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:54:22,926 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:22,926 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:54:22,926 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:54:22,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:54:22,926 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-11-28 03:54:22,926 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-28 03:54:22,926 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:54:22,926 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:22,926 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(399): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-11-28 03:54:22,926 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:22,927 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:22,927 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:22,927 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:54:22,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:54:22,927 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-11-28 03:54:22,927 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-28 03:54:23,027 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-28 03:54:23,028 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-28 03:54:23,032 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-28 03:54:23,034 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-11-28 03:54:23,034 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:23,034 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-11-28 03:54:23,034 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:54:23,034 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-11-28 03:54:23,034 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-11-28 03:54:23,034 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:23,034 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:23,036 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:54:23,036 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:23,036 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:54:23,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:54:23,036 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:23,036 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-28 03:54:23,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:23,037 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:23,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-28 03:54:23,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:23,037 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:23,037 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-11-28 03:54:23,037 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:23,037 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-11-28 03:54:23,037 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:54:23,037 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-11-28 03:54:23,038 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-28 03:54:23,038 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-28 03:54:23,038 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701143657995.a075e19a5494ecfa5418f707da74e5f0. 2023-11-28 03:54:23,038 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701143657995.a075e19a5494ecfa5418f707da74e5f0. started... 2023-11-28 03:54:23,038 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing a075e19a5494ecfa5418f707da74e5f0 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-28 03:54:23,050 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205ff085131ff044824aa6bf8f52de887f5 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151205ff085131ff044824aa6bf8f52de887f5 2023-11-28 03:54:23,455 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a075e19a5494ecfa5418f707da74e5f0/.tmp/family1/827e25cb2f1246c6b375eeb9f15645b0 2023-11-28 03:54:23,460 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a075e19a5494ecfa5418f707da74e5f0/.tmp/family1/827e25cb2f1246c6b375eeb9f15645b0 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a075e19a5494ecfa5418f707da74e5f0/family1/827e25cb2f1246c6b375eeb9f15645b0 2023-11-28 03:54:23,464 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a075e19a5494ecfa5418f707da74e5f0/family1/827e25cb2f1246c6b375eeb9f15645b0, entries=1, sequenceid=17, filesize=5.0 K 2023-11-28 03:54:23,465 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for a075e19a5494ecfa5418f707da74e5f0 in 427ms, sequenceid=17, compaction requested=true 2023-11-28 03:54:23,465 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for a075e19a5494ecfa5418f707da74e5f0: 2023-11-28 03:54:23,465 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701143657995.a075e19a5494ecfa5418f707da74e5f0. 2023-11-28 03:54:23,465 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-28 03:54:23,465 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-28 03:54:23,465 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:23,465 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-11-28 03:54:23,465 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-11-28 03:54:23,468 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:23,468 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:23,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:23,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:54:23,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:54:23,468 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:23,469 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-28 03:54:23,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:54:23,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:54:23,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:23,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:23,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:54:23,470 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-11-28 03:54:23,470 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@17c20a80[Count = 0] remaining members to acquire global barrier 2023-11-28 03:54:23,470 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-11-28 03:54:23,470 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:23,471 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:23,471 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:23,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:23,471 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-11-28 03:54:23,471 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-11-28 03:54:23,471 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,41203,1701143600653' in zk 2023-11-28 03:54:23,472 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:23,472 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-28 03:54:23,473 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:23,473 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-11-28 03:54:23,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:23,473 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:54:23,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:54:23,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:54:23,473 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-11-28 03:54:23,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:54:23,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:54:23,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:23,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:23,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:54:23,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:23,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:23,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,41203,1701143600653': 2023-11-28 03:54:23,475 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-11-28 03:54:23,475 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-11-28 03:54:23,475 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-28 03:54:23,475 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-28 03:54:23,475 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:23,475 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-28 03:54:23,477 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:23,477 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:23,477 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:23,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:23,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:54:23,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:54:23,477 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:54:23,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:23,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:54:23,478 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:54:23,478 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:23,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:54:23,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:23,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:23,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:54:23,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:23,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:23,479 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:23,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:54:23,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:23,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:23,482 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:23,482 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:23,482 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:23,482 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:54:23,482 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:23,482 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-11-28 03:54:23,482 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:54:23,482 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(399): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-11-28 03:54:23,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:54:23,482 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-28 03:54:23,482 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:23,482 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:54:23,482 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:54:23,483 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:23,483 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:23,483 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-11-28 03:54:23,483 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-28 03:54:23,483 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:54:23,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:54:23,583 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-28 03:54:23,584 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-28 03:54:23,588 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-28 03:54:23,590 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-11-28 03:54:23,590 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:23,590 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-11-28 03:54:23,590 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:54:23,590 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-11-28 03:54:23,590 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-11-28 03:54:23,590 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:23,590 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:23,592 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:54:23,592 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:23,592 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:54:23,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:54:23,592 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:23,593 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-28 03:54:23,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:23,593 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:23,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-28 03:54:23,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:23,593 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:23,594 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-11-28 03:54:23,594 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:23,594 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-11-28 03:54:23,594 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:54:23,594 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-11-28 03:54:23,594 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-28 03:54:23,594 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-28 03:54:23,594 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701143657995.a075e19a5494ecfa5418f707da74e5f0. 2023-11-28 03:54:23,594 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701143657995.a075e19a5494ecfa5418f707da74e5f0. started... 2023-11-28 03:54:23,594 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing a075e19a5494ecfa5418f707da74e5f0 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-28 03:54:24,008 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151228fe01862b44214779a02aeb446a01a215 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151228fe01862b44214779a02aeb446a01a215 2023-11-28 03:54:24,414 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a075e19a5494ecfa5418f707da74e5f0/.tmp/family1/39b19a9eb5714cd79ec00a181a1360d3 2023-11-28 03:54:24,419 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a075e19a5494ecfa5418f707da74e5f0/.tmp/family1/39b19a9eb5714cd79ec00a181a1360d3 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a075e19a5494ecfa5418f707da74e5f0/family1/39b19a9eb5714cd79ec00a181a1360d3 2023-11-28 03:54:24,424 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a075e19a5494ecfa5418f707da74e5f0/family1/39b19a9eb5714cd79ec00a181a1360d3, entries=1, sequenceid=21, filesize=5.0 K 2023-11-28 03:54:24,425 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for a075e19a5494ecfa5418f707da74e5f0 in 831ms, sequenceid=21, compaction requested=true 2023-11-28 03:54:24,425 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for a075e19a5494ecfa5418f707da74e5f0: 2023-11-28 03:54:24,425 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701143657995.a075e19a5494ecfa5418f707da74e5f0. 2023-11-28 03:54:24,425 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-28 03:54:24,425 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-28 03:54:24,425 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:24,425 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-11-28 03:54:24,425 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-11-28 03:54:24,429 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:24,429 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:24,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:24,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:54:24,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:54:24,430 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:24,430 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-28 03:54:24,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:54:24,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:54:24,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:24,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:24,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:54:24,431 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-11-28 03:54:24,431 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@13ba75b7[Count = 0] remaining members to acquire global barrier 2023-11-28 03:54:24,431 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-11-28 03:54:24,432 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:24,433 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:24,433 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:24,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:24,433 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-11-28 03:54:24,433 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-11-28 03:54:24,433 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:24,433 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-28 03:54:24,433 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,41203,1701143600653' in zk 2023-11-28 03:54:24,437 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-11-28 03:54:24,437 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:24,437 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:54:24,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:24,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:54:24,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:54:24,437 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-11-28 03:54:24,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:54:24,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:54:24,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:24,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:24,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:54:24,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:24,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:24,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,41203,1701143600653': 2023-11-28 03:54:24,440 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-11-28 03:54:24,440 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-11-28 03:54:24,440 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-28 03:54:24,440 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-28 03:54:24,440 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:24,440 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-28 03:54:24,442 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:24,442 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:24,442 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:24,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:24,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:24,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:54:24,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:54:24,442 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:54:24,442 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:54:24,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:54:24,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:54:24,442 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:24,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:24,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:24,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:54:24,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:24,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:24,444 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:24,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:54:24,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:24,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:24,446 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:24,446 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:24,446 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:24,446 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:54:24,446 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:54:24,446 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:24,446 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:54:24,447 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-11-28 03:54:24,447 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-28 03:54:24,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:54:24,447 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:54:24,447 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:24,447 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:24,447 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:24,447 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-11-28 03:54:24,447 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-28 03:54:24,447 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:54:24,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:54:24,547 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-28 03:54:24,548 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-28 03:54:24,552 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-28 03:54:24,554 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-11-28 03:54:24,554 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:24,554 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-11-28 03:54:24,554 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:54:24,554 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-11-28 03:54:24,554 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-11-28 03:54:24,554 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:24,554 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:24,556 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:24,556 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:54:24,556 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:54:24,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:54:24,556 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:24,556 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-28 03:54:24,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:24,557 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:24,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-28 03:54:24,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:24,557 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:24,557 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-11-28 03:54:24,557 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:24,557 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-11-28 03:54:24,558 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:54:24,558 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-11-28 03:54:24,558 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-28 03:54:24,558 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-28 03:54:24,558 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701143657995.a075e19a5494ecfa5418f707da74e5f0. 2023-11-28 03:54:24,558 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701143657995.a075e19a5494ecfa5418f707da74e5f0. started... 2023-11-28 03:54:24,558 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing a075e19a5494ecfa5418f707da74e5f0 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-28 03:54:24,669 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-11-28 03:54:24,971 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512319bd0d3126f5742d0809f1d885cbc3841 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201512319bd0d3126f5742d0809f1d885cbc3841 2023-11-28 03:54:25,377 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a075e19a5494ecfa5418f707da74e5f0/.tmp/family1/894464ab65ed469fad3389e1ebd1fbfb 2023-11-28 03:54:25,382 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a075e19a5494ecfa5418f707da74e5f0/.tmp/family1/894464ab65ed469fad3389e1ebd1fbfb as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a075e19a5494ecfa5418f707da74e5f0/family1/894464ab65ed469fad3389e1ebd1fbfb 2023-11-28 03:54:25,386 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a075e19a5494ecfa5418f707da74e5f0/family1/894464ab65ed469fad3389e1ebd1fbfb, entries=1, sequenceid=25, filesize=5.0 K 2023-11-28 03:54:25,387 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for a075e19a5494ecfa5418f707da74e5f0 in 829ms, sequenceid=25, compaction requested=true 2023-11-28 03:54:25,387 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for a075e19a5494ecfa5418f707da74e5f0: 2023-11-28 03:54:25,387 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701143657995.a075e19a5494ecfa5418f707da74e5f0. 2023-11-28 03:54:25,387 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-28 03:54:25,387 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-28 03:54:25,387 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:25,387 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-11-28 03:54:25,387 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-11-28 03:54:25,389 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:25,389 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:25,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:25,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:54:25,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:54:25,389 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:25,389 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-28 03:54:25,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:54:25,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:54:25,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:25,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:25,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:54:25,391 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-11-28 03:54:25,391 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@55d5d10c[Count = 0] remaining members to acquire global barrier 2023-11-28 03:54:25,391 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-11-28 03:54:25,391 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:25,392 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:25,392 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:25,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:25,392 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-11-28 03:54:25,392 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-11-28 03:54:25,392 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,41203,1701143600653' in zk 2023-11-28 03:54:25,392 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:25,392 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-28 03:54:25,394 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:25,394 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-11-28 03:54:25,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:25,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:54:25,394 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:54:25,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:54:25,394 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-11-28 03:54:25,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:54:25,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:54:25,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:25,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:25,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:54:25,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:25,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:25,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,41203,1701143600653': 2023-11-28 03:54:25,396 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-11-28 03:54:25,396 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-11-28 03:54:25,396 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-28 03:54:25,396 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-28 03:54:25,396 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:25,396 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-28 03:54:25,399 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:25,399 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:25,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:25,399 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:25,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:54:25,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:25,399 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:54:25,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:54:25,399 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:54:25,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:54:25,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:54:25,400 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:25,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:25,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:25,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:54:25,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:25,401 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:25,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:25,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:54:25,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:25,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:25,403 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:25,403 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:25,403 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:54:25,403 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:25,403 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:54:25,403 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:25,403 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-11-28 03:54:25,403 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(399): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-11-28 03:54:25,403 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:54:25,403 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-28 03:54:25,403 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:25,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:54:25,403 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:54:25,403 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:25,403 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:25,403 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-11-28 03:54:25,404 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-28 03:54:25,404 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:54:25,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:54:25,504 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-28 03:54:25,505 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-28 03:54:25,509 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-28 03:54:25,510 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-11-28 03:54:25,510 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:25,510 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-11-28 03:54:25,510 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:54:25,510 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-11-28 03:54:25,510 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-11-28 03:54:25,511 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:25,511 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:25,512 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:54:25,512 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:25,512 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:54:25,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:54:25,513 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:25,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:25,513 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-28 03:54:25,513 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:25,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-28 03:54:25,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:25,513 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:25,514 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-11-28 03:54:25,514 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:25,514 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-11-28 03:54:25,514 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:54:25,514 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-11-28 03:54:25,514 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-28 03:54:25,514 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-28 03:54:25,514 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701143657995.a075e19a5494ecfa5418f707da74e5f0. 2023-11-28 03:54:25,514 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701143657995.a075e19a5494ecfa5418f707da74e5f0. started... 2023-11-28 03:54:25,514 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing a075e19a5494ecfa5418f707da74e5f0 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-28 03:54:25,927 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601010689b552da1b4c14be7e7c55b804609c to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201601010689b552da1b4c14be7e7c55b804609c 2023-11-28 03:54:26,332 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a075e19a5494ecfa5418f707da74e5f0/.tmp/family1/ac2d1c4b4d78401583e90c1ee3fcf562 2023-11-28 03:54:26,338 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a075e19a5494ecfa5418f707da74e5f0/.tmp/family1/ac2d1c4b4d78401583e90c1ee3fcf562 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a075e19a5494ecfa5418f707da74e5f0/family1/ac2d1c4b4d78401583e90c1ee3fcf562 2023-11-28 03:54:26,343 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a075e19a5494ecfa5418f707da74e5f0/family1/ac2d1c4b4d78401583e90c1ee3fcf562, entries=1, sequenceid=29, filesize=5.0 K 2023-11-28 03:54:26,344 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for a075e19a5494ecfa5418f707da74e5f0 in 830ms, sequenceid=29, compaction requested=true 2023-11-28 03:54:26,344 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for a075e19a5494ecfa5418f707da74e5f0: 2023-11-28 03:54:26,344 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701143657995.a075e19a5494ecfa5418f707da74e5f0. 2023-11-28 03:54:26,344 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-28 03:54:26,344 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-28 03:54:26,344 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:26,344 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-11-28 03:54:26,344 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-11-28 03:54:26,346 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:26,346 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:26,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:26,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:54:26,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:54:26,346 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:26,346 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-28 03:54:26,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:54:26,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:54:26,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:26,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:26,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:54:26,348 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-11-28 03:54:26,348 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@79cb9b8[Count = 0] remaining members to acquire global barrier 2023-11-28 03:54:26,348 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-11-28 03:54:26,348 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:26,353 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:26,353 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:26,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:26,353 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-11-28 03:54:26,354 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-11-28 03:54:26,354 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,41203,1701143600653' in zk 2023-11-28 03:54:26,354 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:26,354 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-28 03:54:26,358 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-11-28 03:54:26,358 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:26,358 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:54:26,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:26,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:54:26,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:54:26,358 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-11-28 03:54:26,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:54:26,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:54:26,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:26,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:26,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:54:26,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:26,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:26,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,41203,1701143600653': 2023-11-28 03:54:26,360 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-11-28 03:54:26,360 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-11-28 03:54:26,360 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-28 03:54:26,360 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-28 03:54:26,360 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:26,360 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-28 03:54:26,362 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:26,362 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:26,362 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:54:26,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:26,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:54:26,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:54:26,362 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:26,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:26,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:54:26,363 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:54:26,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:54:26,363 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:26,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:26,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:26,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:54:26,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:26,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:26,364 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:26,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:54:26,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:26,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:26,367 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:26,367 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:54:26,367 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:26,367 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:54:26,367 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-11-28 03:54:26,367 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-28 03:54:26,367 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:54:26,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:54:26,367 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:54:26,367 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:26,367 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:26,367 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:26,367 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:26,367 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-11-28 03:54:26,367 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:26,367 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-28 03:54:26,367 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:54:26,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:54:26,467 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-28 03:54:26,468 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-28 03:54:26,473 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-28 03:54:26,474 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-11-28 03:54:26,474 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:26,474 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-11-28 03:54:26,474 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:54:26,474 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-11-28 03:54:26,474 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-11-28 03:54:26,475 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:26,475 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:26,476 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:26,476 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:54:26,476 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:54:26,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:54:26,477 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:26,477 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-28 03:54:26,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:26,477 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:26,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-28 03:54:26,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:26,477 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:26,478 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-11-28 03:54:26,478 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:26,478 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-11-28 03:54:26,478 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:54:26,478 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-11-28 03:54:26,478 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-28 03:54:26,478 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-28 03:54:26,478 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701143657995.a075e19a5494ecfa5418f707da74e5f0. 2023-11-28 03:54:26,478 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701143657995.a075e19a5494ecfa5418f707da74e5f0. started... 2023-11-28 03:54:26,478 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing a075e19a5494ecfa5418f707da74e5f0 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-28 03:54:26,893 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2016010313822f521548443aad6dba708272e8ef to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2016010313822f521548443aad6dba708272e8ef 2023-11-28 03:54:27,300 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a075e19a5494ecfa5418f707da74e5f0/.tmp/family1/df7f79657a2f4b169ff5476e7654ebd3 2023-11-28 03:54:27,305 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a075e19a5494ecfa5418f707da74e5f0/.tmp/family1/df7f79657a2f4b169ff5476e7654ebd3 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a075e19a5494ecfa5418f707da74e5f0/family1/df7f79657a2f4b169ff5476e7654ebd3 2023-11-28 03:54:27,309 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a075e19a5494ecfa5418f707da74e5f0/family1/df7f79657a2f4b169ff5476e7654ebd3, entries=1, sequenceid=33, filesize=5.0 K 2023-11-28 03:54:27,310 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for a075e19a5494ecfa5418f707da74e5f0 in 832ms, sequenceid=33, compaction requested=true 2023-11-28 03:54:27,310 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for a075e19a5494ecfa5418f707da74e5f0: 2023-11-28 03:54:27,310 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701143657995.a075e19a5494ecfa5418f707da74e5f0. 2023-11-28 03:54:27,310 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-28 03:54:27,310 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-28 03:54:27,310 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:27,310 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-11-28 03:54:27,310 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-11-28 03:54:27,312 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:27,312 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:27,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:27,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:54:27,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:54:27,312 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:27,313 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-28 03:54:27,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:54:27,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:54:27,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:27,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:27,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:54:27,314 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-11-28 03:54:27,314 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@d35c1[Count = 0] remaining members to acquire global barrier 2023-11-28 03:54:27,314 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-11-28 03:54:27,314 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:27,315 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:27,316 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:27,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:27,316 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-11-28 03:54:27,316 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:27,316 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-11-28 03:54:27,316 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-28 03:54:27,316 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,41203,1701143600653' in zk 2023-11-28 03:54:27,318 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-11-28 03:54:27,318 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:27,319 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:54:27,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:27,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:54:27,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:54:27,319 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-11-28 03:54:27,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:54:27,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:54:27,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:27,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:27,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:54:27,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:27,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:27,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,41203,1701143600653': 2023-11-28 03:54:27,321 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-11-28 03:54:27,321 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-11-28 03:54:27,321 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-28 03:54:27,321 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-28 03:54:27,321 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:27,321 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-28 03:54:27,323 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:27,323 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:27,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:27,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:54:27,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:54:27,323 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:27,323 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:54:27,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:27,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:54:27,324 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:27,324 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:54:27,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:54:27,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:27,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:27,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:54:27,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:27,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:27,325 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:27,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:54:27,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:27,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:27,327 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:27,328 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:27,328 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:54:27,328 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:27,328 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:54:27,328 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(399): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-11-28 03:54:27,328 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:54:27,328 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:54:27,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:54:27,328 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-11-28 03:54:27,328 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-28 03:54:27,328 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:27,328 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:27,328 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:27,328 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:27,328 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-11-28 03:54:27,328 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-28 03:54:27,328 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:54:27,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:54:27,429 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-28 03:54:27,430 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-28 03:54:27,434 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-28 03:54:27,435 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-11-28 03:54:27,435 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:27,435 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-11-28 03:54:27,436 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:54:27,436 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-11-28 03:54:27,436 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-11-28 03:54:27,436 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:27,436 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:27,437 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:54:27,437 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:27,437 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:54:27,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:54:27,438 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:27,438 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-28 03:54:27,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:27,438 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:27,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-28 03:54:27,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:27,439 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:27,439 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-11-28 03:54:27,439 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:27,439 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-11-28 03:54:27,439 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:54:27,439 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-11-28 03:54:27,439 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-28 03:54:27,439 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-28 03:54:27,439 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701143657995.a075e19a5494ecfa5418f707da74e5f0. 2023-11-28 03:54:27,439 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701143657995.a075e19a5494ecfa5418f707da74e5f0. started... 2023-11-28 03:54:27,439 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing a075e19a5494ecfa5418f707da74e5f0 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-28 03:54:27,453 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20150907265b885f776040b991c9511e3d83a031 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20150907265b885f776040b991c9511e3d83a031 2023-11-28 03:54:27,858 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a075e19a5494ecfa5418f707da74e5f0/.tmp/family1/9ffdc14b46ad40009206a254d38205d4 2023-11-28 03:54:27,863 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a075e19a5494ecfa5418f707da74e5f0/.tmp/family1/9ffdc14b46ad40009206a254d38205d4 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a075e19a5494ecfa5418f707da74e5f0/family1/9ffdc14b46ad40009206a254d38205d4 2023-11-28 03:54:27,866 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a075e19a5494ecfa5418f707da74e5f0/family1/9ffdc14b46ad40009206a254d38205d4, entries=1, sequenceid=37, filesize=5.0 K 2023-11-28 03:54:27,867 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for a075e19a5494ecfa5418f707da74e5f0 in 428ms, sequenceid=37, compaction requested=true 2023-11-28 03:54:27,867 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for a075e19a5494ecfa5418f707da74e5f0: 2023-11-28 03:54:27,867 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701143657995.a075e19a5494ecfa5418f707da74e5f0. 2023-11-28 03:54:27,867 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-28 03:54:27,867 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-28 03:54:27,867 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:27,868 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-11-28 03:54:27,868 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-11-28 03:54:27,870 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:27,870 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:27,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:27,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:54:27,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:54:27,871 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:27,871 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-28 03:54:27,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:54:27,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:54:27,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:27,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:27,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:54:27,872 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-11-28 03:54:27,872 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@40daffba[Count = 0] remaining members to acquire global barrier 2023-11-28 03:54:27,872 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-11-28 03:54:27,872 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:27,874 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:27,874 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:27,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:27,874 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:27,874 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-28 03:54:27,874 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-11-28 03:54:27,874 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-11-28 03:54:27,874 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,41203,1701143600653' in zk 2023-11-28 03:54:27,875 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:27,875 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-11-28 03:54:27,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:27,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:54:27,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:54:27,875 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:54:27,876 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-11-28 03:54:27,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:54:27,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:54:27,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:27,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:27,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:54:27,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:27,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:27,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,41203,1701143600653': 2023-11-28 03:54:27,878 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-11-28 03:54:27,878 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-11-28 03:54:27,878 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-28 03:54:27,878 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-28 03:54:27,878 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:27,878 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-28 03:54:27,880 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:27,880 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:27,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:27,880 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:27,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:54:27,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:54:27,880 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:54:27,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:27,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:54:27,881 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:27,881 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:54:27,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:54:27,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:27,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:27,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:54:27,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:27,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:27,882 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:27,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:54:27,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:27,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:27,888 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:27,888 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:54:27,888 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:27,888 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:54:27,888 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:27,888 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:54:27,888 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:54:27,888 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-11-28 03:54:27,888 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-28 03:54:27,888 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:27,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:54:27,888 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:27,888 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:27,888 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:27,889 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-11-28 03:54:27,889 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:54:27,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:54:27,889 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-28 03:54:27,989 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-28 03:54:27,990 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-28 03:54:27,994 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-28 03:54:27,996 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-11-28 03:54:27,996 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:27,996 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-11-28 03:54:27,996 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:54:27,996 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-11-28 03:54:27,996 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-11-28 03:54:27,996 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:27,996 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:27,998 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:27,998 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:54:27,998 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:54:27,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:54:27,998 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:27,998 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-28 03:54:27,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:27,999 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:27,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-28 03:54:27,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:27,999 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:27,999 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-11-28 03:54:27,999 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:27,999 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-11-28 03:54:27,999 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:54:27,999 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-11-28 03:54:27,999 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-28 03:54:27,999 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-28 03:54:28,000 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701143657995.a075e19a5494ecfa5418f707da74e5f0. 2023-11-28 03:54:28,000 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701143657995.a075e19a5494ecfa5418f707da74e5f0. started... 2023-11-28 03:54:28,000 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing a075e19a5494ecfa5418f707da74e5f0 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-28 03:54:28,412 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511202bff21ecee7c4757aa8df42df1ba9665 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201511202bff21ecee7c4757aa8df42df1ba9665 2023-11-28 03:54:28,817 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a075e19a5494ecfa5418f707da74e5f0/.tmp/family1/8b72bc175eb342c59c693d4560d1c6c6 2023-11-28 03:54:28,822 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a075e19a5494ecfa5418f707da74e5f0/.tmp/family1/8b72bc175eb342c59c693d4560d1c6c6 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a075e19a5494ecfa5418f707da74e5f0/family1/8b72bc175eb342c59c693d4560d1c6c6 2023-11-28 03:54:28,826 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a075e19a5494ecfa5418f707da74e5f0/family1/8b72bc175eb342c59c693d4560d1c6c6, entries=1, sequenceid=41, filesize=5.0 K 2023-11-28 03:54:28,827 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for a075e19a5494ecfa5418f707da74e5f0 in 827ms, sequenceid=41, compaction requested=true 2023-11-28 03:54:28,827 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for a075e19a5494ecfa5418f707da74e5f0: 2023-11-28 03:54:28,827 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701143657995.a075e19a5494ecfa5418f707da74e5f0. 2023-11-28 03:54:28,827 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-28 03:54:28,827 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-28 03:54:28,827 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:28,827 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-11-28 03:54:28,827 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-11-28 03:54:28,829 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:28,829 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:28,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:28,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:54:28,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:54:28,829 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:28,829 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-28 03:54:28,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:54:28,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:54:28,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:28,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:28,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:54:28,831 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-11-28 03:54:28,831 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3b7c1b1c[Count = 0] remaining members to acquire global barrier 2023-11-28 03:54:28,831 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-11-28 03:54:28,831 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:28,833 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:28,833 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:28,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:28,833 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-11-28 03:54:28,833 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-11-28 03:54:28,833 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,41203,1701143600653' in zk 2023-11-28 03:54:28,833 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:28,833 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-28 03:54:28,834 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-11-28 03:54:28,834 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:28,834 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:54:28,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:28,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:54:28,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:54:28,835 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-11-28 03:54:28,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:54:28,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:54:28,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:28,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:28,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:54:28,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:28,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:28,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,41203,1701143600653': 2023-11-28 03:54:28,838 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-11-28 03:54:28,838 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-11-28 03:54:28,838 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-28 03:54:28,838 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-28 03:54:28,838 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:28,838 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-28 03:54:28,839 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:28,839 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:28,840 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:54:28,840 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:28,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:28,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:54:28,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:54:28,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:28,840 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:54:28,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:54:28,840 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:28,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:54:28,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:28,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:28,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:54:28,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:28,841 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:28,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:28,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:54:28,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:28,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:28,844 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:28,844 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:28,844 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:28,844 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:54:28,844 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:28,844 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:54:28,844 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:54:28,844 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:28,844 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:28,844 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:28,844 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:54:28,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:54:28,844 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-11-28 03:54:28,844 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-28 03:54:28,844 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(399): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-11-28 03:54:28,845 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:54:28,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:54:28,845 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-11-28 03:54:28,845 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-28 03:54:28,945 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-28 03:54:28,946 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-28 03:54:28,947 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=38593] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:28,949 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] procedure2.ProcedureExecutor(1028): Stored pid=42, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-11-28 03:54:28,950 DEBUG [PEWorker-1] locking.LockProcedure(309): LOCKED pid=42, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-11-28 03:54:28,950 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-11-28 03:54:28,950 INFO [PEWorker-1] procedure2.TimeoutExecutorThread(81): ADDED pid=42, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1701144268950 2023-11-28 03:54:28,953 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-11-28 03:54:28,955 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column 'family1' 2023-11-28 03:54:28,955 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-11-28 03:54:28,957 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] zookeeper.ReadOnlyZKClient(139): Connect 0x4163f506 to 127.0.0.1:55641 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-11-28 03:54:28,964 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] ipc.AbstractRpcClient(189): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@3f183e3, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-11-28 03:54:28,965 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-11-28 03:54:29,826 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601038e1ae496c2444bd19d3bba5cbcb1cb3e to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201601038e1ae496c2444bd19d3bba5cbcb1cb3e 2023-11-28 03:54:29,826 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-11-28 03:54:29,839 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(122): Using SIMPLE authentication for service=ClientService, sasl=false 2023-11-28 03:54:29,840 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:38310, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-11-28 03:54:29,847 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(122): Using SIMPLE authentication for service=MasterService, sasl=false 2023-11-28 03:54:29,849 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:44758, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-11-28 03:54:29,858 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/0bec98f2dcde4667bc63b601c72a8b50 first=Optional[r3] last=Optional[r6] 2023-11-28 03:54:29,858 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701143657995.a075e19a5494ecfa5418f707da74e5f0., hostname=jenkins-hbase4.apache.org,41203,1701143600653, seqNum=2 for row with hfile group [{family1,hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/0bec98f2dcde4667bc63b601c72a8b50}] 2023-11-28 03:54:29,863 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HStore(763): Validating hfile at hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/0bec98f2dcde4667bc63b601c72a8b50 for inclusion in a075e19a5494ecfa5418f707da74e5f0/family1 2023-11-28 03:54:29,866 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HStore(775): HFile bounds: first=r3 last=r6 2023-11-28 03:54:29,867 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HStore(777): Region bounds: first= last= 2023-11-28 03:54:29,867 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HRegion(2446): Flush status journal for a075e19a5494ecfa5418f707da74e5f0: 2023-11-28 03:54:29,868 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/0bec98f2dcde4667bc63b601c72a8b50 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__75ogopl2g9vdsqb0s7ncrlhge7outlg9e3inejunrkc9eh35970m3f0e5li1dsv8/family1/0bec98f2dcde4667bc63b601c72a8b50 2023-11-28 03:54:29,870 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__75ogopl2g9vdsqb0s7ncrlhge7outlg9e3inejunrkc9eh35970m3f0e5li1dsv8/family1/0bec98f2dcde4667bc63b601c72a8b50 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a075e19a5494ecfa5418f707da74e5f0/family1/a607e3e415d24611bc50388739e2a949_SeqId_44_ 2023-11-28 03:54:29,871 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HStore(849): Loaded HFile hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__75ogopl2g9vdsqb0s7ncrlhge7outlg9e3inejunrkc9eh35970m3f0e5li1dsv8/family1/0bec98f2dcde4667bc63b601c72a8b50 into a075e19a5494ecfa5418f707da74e5f0/family1 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a075e19a5494ecfa5418f707da74e5f0/family1/a607e3e415d24611bc50388739e2a949_SeqId_44_ - updating store file list. 2023-11-28 03:54:29,876 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HStore(882): Loaded HFile hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a075e19a5494ecfa5418f707da74e5f0/family1/a607e3e415d24611bc50388739e2a949_SeqId_44_ into a075e19a5494ecfa5418f707da74e5f0/family1 2023-11-28 03:54:29,876 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HStore(855): Successfully loaded hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__75ogopl2g9vdsqb0s7ncrlhge7outlg9e3inejunrkc9eh35970m3f0e5li1dsv8/family1/0bec98f2dcde4667bc63b601c72a8b50 into a075e19a5494ecfa5418f707da74e5f0/family1 (new location: hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a075e19a5494ecfa5418f707da74e5f0/family1/a607e3e415d24611bc50388739e2a949_SeqId_44_) 2023-11-28 03:54:29,877 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__75ogopl2g9vdsqb0s7ncrlhge7outlg9e3inejunrkc9eh35970m3f0e5li1dsv8/family1/0bec98f2dcde4667bc63b601c72a8b50 2023-11-28 03:54:29,877 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-11-28 03:54:29,880 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 51565 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-11-28 03:54:29,880 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HStore(1912): a075e19a5494ecfa5418f707da74e5f0/family1 is initiating minor compaction 2023-11-28 03:54:29,881 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@4a1acc79; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-28 03:54:29,881 INFO [RS:0;jenkins-hbase4:41203-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of a075e19a5494ecfa5418f707da74e5f0/family1 in testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701143657995.a075e19a5494ecfa5418f707da74e5f0. 2023-11-28 03:54:29,881 INFO [RS:0;jenkins-hbase4:41203-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a075e19a5494ecfa5418f707da74e5f0/family1/b882c18da385433b94ad633b72e7a7c5, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a075e19a5494ecfa5418f707da74e5f0/family1/61ed4c90dc4d472d96eef3d76134d373, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a075e19a5494ecfa5418f707da74e5f0/family1/827e25cb2f1246c6b375eeb9f15645b0, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a075e19a5494ecfa5418f707da74e5f0/family1/39b19a9eb5714cd79ec00a181a1360d3, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a075e19a5494ecfa5418f707da74e5f0/family1/894464ab65ed469fad3389e1ebd1fbfb, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a075e19a5494ecfa5418f707da74e5f0/family1/ac2d1c4b4d78401583e90c1ee3fcf562, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a075e19a5494ecfa5418f707da74e5f0/family1/df7f79657a2f4b169ff5476e7654ebd3, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a075e19a5494ecfa5418f707da74e5f0/family1/9ffdc14b46ad40009206a254d38205d4, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a075e19a5494ecfa5418f707da74e5f0/family1/8b72bc175eb342c59c693d4560d1c6c6, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a075e19a5494ecfa5418f707da74e5f0/family1/a607e3e415d24611bc50388739e2a949_SeqId_44_] into tmpdir=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a075e19a5494ecfa5418f707da74e5f0/.tmp, totalSize=50.4 K 2023-11-28 03:54:29,881 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] compactions.Compactor(207): Compacting b882c18da385433b94ad633b72e7a7c5, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-11-28 03:54:29,881 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] compactions.Compactor(207): Compacting 61ed4c90dc4d472d96eef3d76134d373, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=13 2023-11-28 03:54:29,881 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] compactions.Compactor(207): Compacting 827e25cb2f1246c6b375eeb9f15645b0, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=17 2023-11-28 03:54:29,881 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] compactions.Compactor(207): Compacting 39b19a9eb5714cd79ec00a181a1360d3, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=21 2023-11-28 03:54:29,881 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] compactions.Compactor(207): Compacting 894464ab65ed469fad3389e1ebd1fbfb, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=25 2023-11-28 03:54:29,881 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] compactions.Compactor(207): Compacting ac2d1c4b4d78401583e90c1ee3fcf562, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=29 2023-11-28 03:54:29,881 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] compactions.Compactor(207): Compacting df7f79657a2f4b169ff5476e7654ebd3, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33 2023-11-28 03:54:29,881 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] compactions.Compactor(207): Compacting 9ffdc14b46ad40009206a254d38205d4, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=37 2023-11-28 03:54:29,882 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] compactions.Compactor(207): Compacting 8b72bc175eb342c59c693d4560d1c6c6, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=41 2023-11-28 03:54:29,882 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] compactions.Compactor(207): Compacting a607e3e415d24611bc50388739e2a949_SeqId_44_, keycount=4, bloomtype=ROW, size=5.6 K, encoding=NONE, compression=NONE, seqNum=45 2023-11-28 03:54:29,882 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-11-28 03:54:29,883 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-28 03:54:29,885 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151228fe01862b44214779a02aeb446a01a215 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151228fe01862b44214779a02aeb446a01a215 2023-11-28 03:54:29,887 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201512319bd0d3126f5742d0809f1d885cbc3841 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201512319bd0d3126f5742d0809f1d885cbc3841 2023-11-28 03:54:29,888 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201601010689b552da1b4c14be7e7c55b804609c to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201601010689b552da1b4c14be7e7c55b804609c 2023-11-28 03:54:29,889 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2016010313822f521548443aad6dba708272e8ef to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2016010313822f521548443aad6dba708272e8ef 2023-11-28 03:54:29,889 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 4 expired mob files are deleted 2023-11-28 03:54:29,889 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151228' 2023-11-28 03:54:29,890 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-11-28 03:54:29,890 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-11-28 03:54:29,890 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-11-28 03:54:29,890 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-11-28 03:54:29,890 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-11-28 03:54:29,890 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-11-28 03:54:29,890 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-11-28 03:54:29,890 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-11-28 03:54:29,907 INFO [RS:0;jenkins-hbase4:41203-shortCompactions-0] throttle.PressureAwareThroughputController(145): a075e19a5494ecfa5418f707da74e5f0#family1#compaction#60 average throughput is 0.53 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-11-28 03:54:30,724 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a075e19a5494ecfa5418f707da74e5f0/.tmp/family1/108541b512384a1b98c56d5728ce2ab6 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a075e19a5494ecfa5418f707da74e5f0/family1/108541b512384a1b98c56d5728ce2ab6 2023-11-28 03:54:30,729 INFO [RS:0;jenkins-hbase4:41203-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in a075e19a5494ecfa5418f707da74e5f0/family1 of a075e19a5494ecfa5418f707da74e5f0 into 108541b512384a1b98c56d5728ce2ab6(size=6.8 K), total size for store is 198.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-11-28 03:54:30,729 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for a075e19a5494ecfa5418f707da74e5f0: 2023-11-28 03:54:30,729 INFO [RS:0;jenkins-hbase4:41203-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701143657995.a075e19a5494ecfa5418f707da74e5f0., storeName=family1, fileCount=10, fileSize=50.4 K (5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.6 K), priority=2, time=1701143669880; duration=0sec 2023-11-28 03:54:30,729 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-28 03:54:30,733 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205d5dcb43b1dbe444cb1db07aa35993136 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151205d5dcb43b1dbe444cb1db07aa35993136 2023-11-28 03:54:30,733 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-11-28 03:54:30,757 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/69044a5260eb4c87aedf8e4d1e55e43a first=Optional[r00] last=Optional[r2] 2023-11-28 03:54:30,758 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701143657995.a075e19a5494ecfa5418f707da74e5f0., hostname=jenkins-hbase4.apache.org,41203,1701143600653, seqNum=2 for row with hfile group [{family1,hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/69044a5260eb4c87aedf8e4d1e55e43a}] 2023-11-28 03:54:30,760 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HStore(763): Validating hfile at hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/69044a5260eb4c87aedf8e4d1e55e43a for inclusion in a075e19a5494ecfa5418f707da74e5f0/family1 2023-11-28 03:54:30,762 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HStore(775): HFile bounds: first=r00 last=r2 2023-11-28 03:54:30,762 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HStore(777): Region bounds: first= last= 2023-11-28 03:54:30,762 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HRegion(2446): Flush status journal for a075e19a5494ecfa5418f707da74e5f0: 2023-11-28 03:54:30,763 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/69044a5260eb4c87aedf8e4d1e55e43a to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__6ig2oudi6n0bb9nvevc6ksrt20120drhbk5ed074kfc69d18s28uqcdjp40klbr9/family1/69044a5260eb4c87aedf8e4d1e55e43a 2023-11-28 03:54:30,766 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__6ig2oudi6n0bb9nvevc6ksrt20120drhbk5ed074kfc69d18s28uqcdjp40klbr9/family1/69044a5260eb4c87aedf8e4d1e55e43a as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a075e19a5494ecfa5418f707da74e5f0/family1/aadeb504cfd047c2811ee7fd66edbcaf_SeqId_47_ 2023-11-28 03:54:30,766 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HStore(849): Loaded HFile hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__6ig2oudi6n0bb9nvevc6ksrt20120drhbk5ed074kfc69d18s28uqcdjp40klbr9/family1/69044a5260eb4c87aedf8e4d1e55e43a into a075e19a5494ecfa5418f707da74e5f0/family1 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a075e19a5494ecfa5418f707da74e5f0/family1/aadeb504cfd047c2811ee7fd66edbcaf_SeqId_47_ - updating store file list. 2023-11-28 03:54:30,769 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HStore(882): Loaded HFile hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a075e19a5494ecfa5418f707da74e5f0/family1/aadeb504cfd047c2811ee7fd66edbcaf_SeqId_47_ into a075e19a5494ecfa5418f707da74e5f0/family1 2023-11-28 03:54:30,769 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HStore(855): Successfully loaded hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__6ig2oudi6n0bb9nvevc6ksrt20120drhbk5ed074kfc69d18s28uqcdjp40klbr9/family1/69044a5260eb4c87aedf8e4d1e55e43a into a075e19a5494ecfa5418f707da74e5f0/family1 (new location: hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a075e19a5494ecfa5418f707da74e5f0/family1/aadeb504cfd047c2811ee7fd66edbcaf_SeqId_47_) 2023-11-28 03:54:30,770 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__6ig2oudi6n0bb9nvevc6ksrt20120drhbk5ed074kfc69d18s28uqcdjp40klbr9/family1/69044a5260eb4c87aedf8e4d1e55e43a 2023-11-28 03:54:30,770 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-11-28 03:54:30,771 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 399433 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-11-28 03:54:30,771 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HStore(1912): a075e19a5494ecfa5418f707da74e5f0/family1 is initiating minor compaction (all files) 2023-11-28 03:54:30,771 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@709fee73; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-28 03:54:30,771 INFO [RS:0;jenkins-hbase4:41203-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of a075e19a5494ecfa5418f707da74e5f0/family1 in testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701143657995.a075e19a5494ecfa5418f707da74e5f0. 2023-11-28 03:54:30,771 INFO [RS:0;jenkins-hbase4:41203-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a075e19a5494ecfa5418f707da74e5f0/family1/633e489f34d44208a9e77c96ae35ea69, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a075e19a5494ecfa5418f707da74e5f0/family1/108541b512384a1b98c56d5728ce2ab6, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a075e19a5494ecfa5418f707da74e5f0/family1/aadeb504cfd047c2811ee7fd66edbcaf_SeqId_47_] into tmpdir=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a075e19a5494ecfa5418f707da74e5f0/.tmp, totalSize=390.1 K 2023-11-28 03:54:30,772 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] compactions.Compactor(207): Compacting 633e489f34d44208a9e77c96ae35ea69, keycount=1000, bloomtype=ROW, size=191.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878824053 2023-11-28 03:54:30,772 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] compactions.Compactor(207): Compacting 108541b512384a1b98c56d5728ce2ab6, keycount=9, bloomtype=ROW, size=6.8 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621224053 2023-11-28 03:54:30,773 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] compactions.Compactor(207): Compacting aadeb504cfd047c2811ee7fd66edbcaf_SeqId_47_, keycount=1002, bloomtype=ROW, size=191.8 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448878824053 2023-11-28 03:54:30,778 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-11-28 03:54:30,784 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a075e19a5494ecfa5418f707da74e5f0/family1/b882c18da385433b94ad633b72e7a7c5, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a075e19a5494ecfa5418f707da74e5f0/family1/61ed4c90dc4d472d96eef3d76134d373, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a075e19a5494ecfa5418f707da74e5f0/family1/827e25cb2f1246c6b375eeb9f15645b0, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a075e19a5494ecfa5418f707da74e5f0/family1/39b19a9eb5714cd79ec00a181a1360d3, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a075e19a5494ecfa5418f707da74e5f0/family1/894464ab65ed469fad3389e1ebd1fbfb, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a075e19a5494ecfa5418f707da74e5f0/family1/ac2d1c4b4d78401583e90c1ee3fcf562, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a075e19a5494ecfa5418f707da74e5f0/family1/df7f79657a2f4b169ff5476e7654ebd3, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a075e19a5494ecfa5418f707da74e5f0/family1/9ffdc14b46ad40009206a254d38205d4, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a075e19a5494ecfa5418f707da74e5f0/family1/8b72bc175eb342c59c693d4560d1c6c6, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a075e19a5494ecfa5418f707da74e5f0/family1/a607e3e415d24611bc50388739e2a949_SeqId_44_] to archive 2023-11-28 03:54:30,785 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-28 03:54:30,787 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-28 03:54:30,788 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a075e19a5494ecfa5418f707da74e5f0/family1/b882c18da385433b94ad633b72e7a7c5 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a075e19a5494ecfa5418f707da74e5f0/family1/b882c18da385433b94ad633b72e7a7c5 2023-11-28 03:54:30,789 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015113084aeaab323b645c7834083383ceec7d6 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015113084aeaab323b645c7834083383ceec7d6 2023-11-28 03:54:30,790 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a075e19a5494ecfa5418f707da74e5f0/family1/61ed4c90dc4d472d96eef3d76134d373 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a075e19a5494ecfa5418f707da74e5f0/family1/61ed4c90dc4d472d96eef3d76134d373 2023-11-28 03:54:30,790 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201512015d75b11b3f36416eb8f97fa9f88a6984 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201512015d75b11b3f36416eb8f97fa9f88a6984 2023-11-28 03:54:30,792 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a075e19a5494ecfa5418f707da74e5f0/family1/827e25cb2f1246c6b375eeb9f15645b0 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a075e19a5494ecfa5418f707da74e5f0/family1/827e25cb2f1246c6b375eeb9f15645b0 2023-11-28 03:54:30,792 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151205ff085131ff044824aa6bf8f52de887f5 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151205ff085131ff044824aa6bf8f52de887f5 2023-11-28 03:54:30,792 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 3 expired mob files are deleted 2023-11-28 03:54:30,792 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151130' 2023-11-28 03:54:30,793 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-11-28 03:54:30,794 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] compactions.PartitionedMobCompactor(354): After compaction, there are 5 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-11-28 03:54:30,794 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-11-28 03:54:30,795 DEBUG [PEWorker-2] locking.LockProcedure(242): UNLOCKED pid=42, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-11-28 03:54:30,795 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a075e19a5494ecfa5418f707da74e5f0/family1/39b19a9eb5714cd79ec00a181a1360d3 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a075e19a5494ecfa5418f707da74e5f0/family1/39b19a9eb5714cd79ec00a181a1360d3 2023-11-28 03:54:30,797 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=42, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE in 1.8470 sec 2023-11-28 03:54:30,798 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a075e19a5494ecfa5418f707da74e5f0/family1/894464ab65ed469fad3389e1ebd1fbfb to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a075e19a5494ecfa5418f707da74e5f0/family1/894464ab65ed469fad3389e1ebd1fbfb 2023-11-28 03:54:30,800 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a075e19a5494ecfa5418f707da74e5f0/family1/ac2d1c4b4d78401583e90c1ee3fcf562 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a075e19a5494ecfa5418f707da74e5f0/family1/ac2d1c4b4d78401583e90c1ee3fcf562 2023-11-28 03:54:30,802 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a075e19a5494ecfa5418f707da74e5f0/family1/df7f79657a2f4b169ff5476e7654ebd3 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a075e19a5494ecfa5418f707da74e5f0/family1/df7f79657a2f4b169ff5476e7654ebd3 2023-11-28 03:54:30,804 INFO [RS:0;jenkins-hbase4:41203-shortCompactions-0] throttle.PressureAwareThroughputController(145): a075e19a5494ecfa5418f707da74e5f0#family1#compaction#61 average throughput is 18.06 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-11-28 03:54:30,808 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a075e19a5494ecfa5418f707da74e5f0/family1/9ffdc14b46ad40009206a254d38205d4 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a075e19a5494ecfa5418f707da74e5f0/family1/9ffdc14b46ad40009206a254d38205d4 2023-11-28 03:54:30,809 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a075e19a5494ecfa5418f707da74e5f0/family1/8b72bc175eb342c59c693d4560d1c6c6 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a075e19a5494ecfa5418f707da74e5f0/family1/8b72bc175eb342c59c693d4560d1c6c6 2023-11-28 03:54:30,810 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a075e19a5494ecfa5418f707da74e5f0/family1/a607e3e415d24611bc50388739e2a949_SeqId_44_ to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a075e19a5494ecfa5418f707da74e5f0/family1/a607e3e415d24611bc50388739e2a949_SeqId_44_ 2023-11-28 03:54:30,878 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.HMaster$12(2454): Client=jenkins//172.31.14.131 modify table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly from 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} to 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} 2023-11-28 03:54:30,879 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] procedure2.ProcedureExecutor(1028): Stored pid=43, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:31,289 DEBUG [PEWorker-5] util.FSTableDescriptors(570): Wrote into hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000002 2023-11-28 03:54:31,290 DEBUG [PEWorker-5] util.FSTableDescriptors(519): Deleted hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000001 2023-11-28 03:54:31,290 INFO [PEWorker-5] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000002 2023-11-28 03:54:31,292 INFO [PEWorker-5] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=44, ppid=43, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly}] 2023-11-28 03:54:31,295 INFO [PEWorker-4] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=a075e19a5494ecfa5418f707da74e5f0, REOPEN/MOVE}] 2023-11-28 03:54:31,295 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=a075e19a5494ecfa5418f707da74e5f0, REOPEN/MOVE 2023-11-28 03:54:31,296 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=a075e19a5494ecfa5418f707da74e5f0, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:31,296 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701143657995.a075e19a5494ecfa5418f707da74e5f0.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1701143671296"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1701143671296"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1701143671296"}]},"ts":"1701143671296"} 2023-11-28 03:54:31,297 INFO [PEWorker-3] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure a075e19a5494ecfa5418f707da74e5f0, server=jenkins-hbase4.apache.org,41203,1701143600653}] 2023-11-28 03:54:31,449 DEBUG [RSProcedureDispatcher-pool-1] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:31,449 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close a075e19a5494ecfa5418f707da74e5f0 2023-11-28 03:54:31,449 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing a075e19a5494ecfa5418f707da74e5f0, disabling compactions & flushes 2023-11-28 03:54:31,449 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1858): waiting for 1 compactions to complete for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701143657995.a075e19a5494ecfa5418f707da74e5f0. 2023-11-28 03:54:31,629 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a075e19a5494ecfa5418f707da74e5f0/.tmp/family1/bbfa095caba54308a337ffd047506f00 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a075e19a5494ecfa5418f707da74e5f0/family1/bbfa095caba54308a337ffd047506f00 2023-11-28 03:54:31,633 INFO [RS:0;jenkins-hbase4:41203-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in a075e19a5494ecfa5418f707da74e5f0/family1 of a075e19a5494ecfa5418f707da74e5f0 into bbfa095caba54308a337ffd047506f00(size=193.0 K), total size for store is 193.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-11-28 03:54:31,633 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for a075e19a5494ecfa5418f707da74e5f0: 2023-11-28 03:54:31,633 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701143657995.a075e19a5494ecfa5418f707da74e5f0. 2023-11-28 03:54:31,633 INFO [RS:0;jenkins-hbase4:41203-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701143657995.a075e19a5494ecfa5418f707da74e5f0., storeName=family1, fileCount=3, fileSize=390.1 K (191.4 K, 6.8 K, 191.8 K), priority=2, time=1701143670771; duration=0sec 2023-11-28 03:54:31,633 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701143657995.a075e19a5494ecfa5418f707da74e5f0. 2023-11-28 03:54:31,633 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-28 03:54:31,633 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701143657995.a075e19a5494ecfa5418f707da74e5f0. after waiting 0 ms 2023-11-28 03:54:31,634 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701143657995.a075e19a5494ecfa5418f707da74e5f0. 2023-11-28 03:54:31,634 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701143657995.a075e19a5494ecfa5418f707da74e5f0.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a075e19a5494ecfa5418f707da74e5f0/family1/633e489f34d44208a9e77c96ae35ea69, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a075e19a5494ecfa5418f707da74e5f0/family1/108541b512384a1b98c56d5728ce2ab6, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a075e19a5494ecfa5418f707da74e5f0/family1/aadeb504cfd047c2811ee7fd66edbcaf_SeqId_47_] to archive 2023-11-28 03:54:31,634 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701143657995.a075e19a5494ecfa5418f707da74e5f0.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-11-28 03:54:31,636 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701143657995.a075e19a5494ecfa5418f707da74e5f0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a075e19a5494ecfa5418f707da74e5f0/family1/633e489f34d44208a9e77c96ae35ea69 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a075e19a5494ecfa5418f707da74e5f0/family1/633e489f34d44208a9e77c96ae35ea69 2023-11-28 03:54:31,637 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701143657995.a075e19a5494ecfa5418f707da74e5f0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a075e19a5494ecfa5418f707da74e5f0/family1/108541b512384a1b98c56d5728ce2ab6 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a075e19a5494ecfa5418f707da74e5f0/family1/108541b512384a1b98c56d5728ce2ab6 2023-11-28 03:54:31,638 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701143657995.a075e19a5494ecfa5418f707da74e5f0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a075e19a5494ecfa5418f707da74e5f0/family1/aadeb504cfd047c2811ee7fd66edbcaf_SeqId_47_ to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a075e19a5494ecfa5418f707da74e5f0/family1/aadeb504cfd047c2811ee7fd66edbcaf_SeqId_47_ 2023-11-28 03:54:31,643 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a075e19a5494ecfa5418f707da74e5f0/recovered.edits/50.seqid, newMaxSeqId=50, maxSeqId=1 2023-11-28 03:54:31,644 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701143657995.a075e19a5494ecfa5418f707da74e5f0. 2023-11-28 03:54:31,644 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for a075e19a5494ecfa5418f707da74e5f0: 2023-11-28 03:54:31,644 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3512): Not adding moved region record: a075e19a5494ecfa5418f707da74e5f0 to self. 2023-11-28 03:54:31,645 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed a075e19a5494ecfa5418f707da74e5f0 2023-11-28 03:54:31,646 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=a075e19a5494ecfa5418f707da74e5f0, regionState=CLOSED 2023-11-28 03:54:31,646 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701143657995.a075e19a5494ecfa5418f707da74e5f0.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1701143671645"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1701143671645"}]},"ts":"1701143671645"} 2023-11-28 03:54:31,648 INFO [PEWorker-2] procedure2.ProcedureExecutor(1825): Finished subprocedure pid=46, resume processing ppid=45 2023-11-28 03:54:31,648 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure a075e19a5494ecfa5418f707da74e5f0, server=jenkins-hbase4.apache.org,41203,1701143600653 in 350 msec 2023-11-28 03:54:31,649 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=a075e19a5494ecfa5418f707da74e5f0, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,41203,1701143600653; forceNewPlan=false, retain=true 2023-11-28 03:54:31,799 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=a075e19a5494ecfa5418f707da74e5f0, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:31,799 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701143657995.a075e19a5494ecfa5418f707da74e5f0.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1701143671799"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1701143671799"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1701143671799"}]},"ts":"1701143671799"} 2023-11-28 03:54:31,801 INFO [PEWorker-4] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=47, ppid=45, state=RUNNABLE; OpenRegionProcedure a075e19a5494ecfa5418f707da74e5f0, server=jenkins-hbase4.apache.org,41203,1701143600653}] 2023-11-28 03:54:31,953 DEBUG [RSProcedureDispatcher-pool-2] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:31,956 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701143657995.a075e19a5494ecfa5418f707da74e5f0. 2023-11-28 03:54:31,956 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => a075e19a5494ecfa5418f707da74e5f0, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701143657995.a075e19a5494ecfa5418f707da74e5f0.', STARTKEY => '', ENDKEY => ''} 2023-11-28 03:54:31,957 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly a075e19a5494ecfa5418f707da74e5f0 2023-11-28 03:54:31,957 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701143657995.a075e19a5494ecfa5418f707da74e5f0.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-28 03:54:31,957 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for a075e19a5494ecfa5418f707da74e5f0 2023-11-28 03:54:31,957 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for a075e19a5494ecfa5418f707da74e5f0 2023-11-28 03:54:31,958 INFO [StoreOpener-a075e19a5494ecfa5418f707da74e5f0-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region a075e19a5494ecfa5418f707da74e5f0 2023-11-28 03:54:31,959 DEBUG [StoreOpener-a075e19a5494ecfa5418f707da74e5f0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a075e19a5494ecfa5418f707da74e5f0/family1 2023-11-28 03:54:31,959 DEBUG [StoreOpener-a075e19a5494ecfa5418f707da74e5f0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a075e19a5494ecfa5418f707da74e5f0/family1 2023-11-28 03:54:31,960 INFO [StoreOpener-a075e19a5494ecfa5418f707da74e5f0-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a075e19a5494ecfa5418f707da74e5f0 columnFamilyName family1 2023-11-28 03:54:31,965 DEBUG [StoreOpener-a075e19a5494ecfa5418f707da74e5f0-1] regionserver.HStore(539): loaded hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a075e19a5494ecfa5418f707da74e5f0/family1/bbfa095caba54308a337ffd047506f00 2023-11-28 03:54:31,965 INFO [StoreOpener-a075e19a5494ecfa5418f707da74e5f0-1] regionserver.HStore(310): Store=a075e19a5494ecfa5418f707da74e5f0/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-28 03:54:31,966 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a075e19a5494ecfa5418f707da74e5f0 2023-11-28 03:54:31,967 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a075e19a5494ecfa5418f707da74e5f0 2023-11-28 03:54:31,969 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for a075e19a5494ecfa5418f707da74e5f0 2023-11-28 03:54:31,970 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened a075e19a5494ecfa5418f707da74e5f0; next sequenceid=51; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60885779, jitterRate=-0.09273119270801544}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-11-28 03:54:31,970 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for a075e19a5494ecfa5418f707da74e5f0: 2023-11-28 03:54:31,971 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2339): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701143657995.a075e19a5494ecfa5418f707da74e5f0., pid=47, masterSystemTime=1701143671953 2023-11-28 03:54:31,973 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2366): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701143657995.a075e19a5494ecfa5418f707da74e5f0. 2023-11-28 03:54:31,973 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701143657995.a075e19a5494ecfa5418f707da74e5f0. 2023-11-28 03:54:31,973 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=a075e19a5494ecfa5418f707da74e5f0, regionState=OPEN, openSeqNum=51, regionLocation=jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:31,973 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701143657995.a075e19a5494ecfa5418f707da74e5f0.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1701143671973"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1701143671973"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1701143671973"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1701143671973"}]},"ts":"1701143671973"} 2023-11-28 03:54:31,976 INFO [PEWorker-1] procedure2.ProcedureExecutor(1825): Finished subprocedure pid=47, resume processing ppid=45 2023-11-28 03:54:31,976 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=47, ppid=45, state=SUCCESS; OpenRegionProcedure a075e19a5494ecfa5418f707da74e5f0, server=jenkins-hbase4.apache.org,41203,1701143600653 in 174 msec 2023-11-28 03:54:31,978 INFO [PEWorker-2] procedure2.ProcedureExecutor(1825): Finished subprocedure pid=45, resume processing ppid=44 2023-11-28 03:54:31,978 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=a075e19a5494ecfa5418f707da74e5f0, REOPEN/MOVE in 681 msec 2023-11-28 03:54:31,980 INFO [PEWorker-5] procedure2.ProcedureExecutor(1825): Finished subprocedure pid=44, resume processing ppid=43 2023-11-28 03:54:31,981 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=44, ppid=43, state=SUCCESS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly in 687 msec 2023-11-28 03:54:31,982 INFO [PEWorker-4] procedure2.ProcedureExecutor(1409): Finished pid=43, state=SUCCESS; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly in 1.1030 sec 2023-11-28 03:54:31,983 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(1230): Checking to see if procedure is done pid=43 2023-11-28 03:54:31,984 INFO [Listener at localhost/42853] compactions.TestMobCompactor(278): alter status finished 2023-11-28 03:54:31,985 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=38593] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:31,986 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] procedure2.ProcedureExecutor(1028): Stored pid=48, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-11-28 03:54:31,987 DEBUG [PEWorker-3] locking.LockProcedure(309): LOCKED pid=48, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-11-28 03:54:31,988 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-11-28 03:54:31,988 INFO [PEWorker-3] procedure2.TimeoutExecutorThread(81): ADDED pid=48, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1701144271988 2023-11-28 03:54:31,990 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-11-28 03:54:31,991 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 5 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column 'family1' 2023-11-28 03:54:31,991 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-11-28 03:54:31,993 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] zookeeper.ReadOnlyZKClient(139): Connect 0x0b70e2fa to 127.0.0.1:55641 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-11-28 03:54:31,998 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] ipc.AbstractRpcClient(189): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@49196ec, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-11-28 03:54:31,998 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-11-28 03:54:31,998 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-11-28 03:54:31,998 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-11-28 03:54:31,998 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-11-28 03:54:31,998 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-11-28 03:54:32,832 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128d9ed3b7a312145d09870e65cc54fbfc2 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151128d9ed3b7a312145d09870e65cc54fbfc2 2023-11-28 03:54:32,833 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-11-28 03:54:32,853 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(122): Using SIMPLE authentication for service=ClientService, sasl=false 2023-11-28 03:54:32,854 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:35910, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-11-28 03:54:32,861 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(122): Using SIMPLE authentication for service=MasterService, sasl=false 2023-11-28 03:54:32,863 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:47798, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-11-28 03:54:32,871 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/aea5f9fa73bd407c9449ad9d00e445fe first=Optional[r05] last=Optional[r8] 2023-11-28 03:54:32,871 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701143657995.a075e19a5494ecfa5418f707da74e5f0., hostname=jenkins-hbase4.apache.org,41203,1701143600653, seqNum=51 for row with hfile group [{family1,hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/aea5f9fa73bd407c9449ad9d00e445fe}] 2023-11-28 03:54:32,876 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HStore(763): Validating hfile at hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/aea5f9fa73bd407c9449ad9d00e445fe for inclusion in a075e19a5494ecfa5418f707da74e5f0/family1 2023-11-28 03:54:32,878 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HStore(775): HFile bounds: first=r05 last=r8 2023-11-28 03:54:32,878 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HStore(777): Region bounds: first= last= 2023-11-28 03:54:32,879 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HRegion(2446): Flush status journal for a075e19a5494ecfa5418f707da74e5f0: 2023-11-28 03:54:32,879 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/aea5f9fa73bd407c9449ad9d00e445fe to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__e4bh2su1sq066r7oka5iq4ji1li44tggpjca4s58u5urmcqbibkeofqoogvfqbvv/family1/aea5f9fa73bd407c9449ad9d00e445fe 2023-11-28 03:54:32,882 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__e4bh2su1sq066r7oka5iq4ji1li44tggpjca4s58u5urmcqbibkeofqoogvfqbvv/family1/aea5f9fa73bd407c9449ad9d00e445fe as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a075e19a5494ecfa5418f707da74e5f0/family1/ecdce964143d4d76961d70a907097fcb_SeqId_53_ 2023-11-28 03:54:32,882 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HStore(849): Loaded HFile hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__e4bh2su1sq066r7oka5iq4ji1li44tggpjca4s58u5urmcqbibkeofqoogvfqbvv/family1/aea5f9fa73bd407c9449ad9d00e445fe into a075e19a5494ecfa5418f707da74e5f0/family1 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a075e19a5494ecfa5418f707da74e5f0/family1/ecdce964143d4d76961d70a907097fcb_SeqId_53_ - updating store file list. 2023-11-28 03:54:32,886 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HStore(882): Loaded HFile hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a075e19a5494ecfa5418f707da74e5f0/family1/ecdce964143d4d76961d70a907097fcb_SeqId_53_ into a075e19a5494ecfa5418f707da74e5f0/family1 2023-11-28 03:54:32,886 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HStore(855): Successfully loaded hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__e4bh2su1sq066r7oka5iq4ji1li44tggpjca4s58u5urmcqbibkeofqoogvfqbvv/family1/aea5f9fa73bd407c9449ad9d00e445fe into a075e19a5494ecfa5418f707da74e5f0/family1 (new location: hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a075e19a5494ecfa5418f707da74e5f0/family1/ecdce964143d4d76961d70a907097fcb_SeqId_53_) 2023-11-28 03:54:32,886 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__e4bh2su1sq066r7oka5iq4ji1li44tggpjca4s58u5urmcqbibkeofqoogvfqbvv/family1/aea5f9fa73bd407c9449ad9d00e445fe 2023-11-28 03:54:32,888 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-11-28 03:54:32,889 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-28 03:54:32,890 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201511202bff21ecee7c4757aa8df42df1ba9665 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201511202bff21ecee7c4757aa8df42df1ba9665 2023-11-28 03:54:32,891 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015112884d93886d19c445b8c9d84378b7f7802 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015112884d93886d19c445b8c9d84378b7f7802 2023-11-28 03:54:32,891 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2 expired mob files are deleted 2023-11-28 03:54:32,891 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151101' 2023-11-28 03:54:32,891 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-11-28 03:54:32,891 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-11-28 03:54:32,892 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-11-28 03:54:32,892 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-11-28 03:54:32,892 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-11-28 03:54:32,893 DEBUG [PEWorker-1] locking.LockProcedure(242): UNLOCKED pid=48, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-11-28 03:54:32,894 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=48, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE in 907 msec 2023-11-28 03:54:32,941 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.HMaster$12(2454): Client=jenkins//172.31.14.131 modify table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly from 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} to 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-11-28 03:54:32,942 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] procedure2.ProcedureExecutor(1028): Stored pid=49, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:33,352 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000003 2023-11-28 03:54:33,353 DEBUG [PEWorker-2] util.FSTableDescriptors(519): Deleted hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000002 2023-11-28 03:54:33,353 INFO [PEWorker-2] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000003 2023-11-28 03:54:33,355 INFO [PEWorker-2] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=50, ppid=49, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly}] 2023-11-28 03:54:33,359 INFO [PEWorker-5] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=51, ppid=50, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=a075e19a5494ecfa5418f707da74e5f0, REOPEN/MOVE}] 2023-11-28 03:54:33,360 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=51, ppid=50, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=a075e19a5494ecfa5418f707da74e5f0, REOPEN/MOVE 2023-11-28 03:54:33,361 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=51 updating hbase:meta row=a075e19a5494ecfa5418f707da74e5f0, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:33,361 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701143657995.a075e19a5494ecfa5418f707da74e5f0.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1701143673361"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1701143673361"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1701143673361"}]},"ts":"1701143673361"} 2023-11-28 03:54:33,362 INFO [PEWorker-4] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=52, ppid=51, state=RUNNABLE; CloseRegionProcedure a075e19a5494ecfa5418f707da74e5f0, server=jenkins-hbase4.apache.org,41203,1701143600653}] 2023-11-28 03:54:33,514 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:33,514 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close a075e19a5494ecfa5418f707da74e5f0 2023-11-28 03:54:33,514 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing a075e19a5494ecfa5418f707da74e5f0, disabling compactions & flushes 2023-11-28 03:54:33,514 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701143657995.a075e19a5494ecfa5418f707da74e5f0. 2023-11-28 03:54:33,514 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701143657995.a075e19a5494ecfa5418f707da74e5f0. 2023-11-28 03:54:33,515 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701143657995.a075e19a5494ecfa5418f707da74e5f0. after waiting 0 ms 2023-11-28 03:54:33,515 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701143657995.a075e19a5494ecfa5418f707da74e5f0. 2023-11-28 03:54:33,520 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a075e19a5494ecfa5418f707da74e5f0/recovered.edits/55.seqid, newMaxSeqId=55, maxSeqId=50 2023-11-28 03:54:33,521 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701143657995.a075e19a5494ecfa5418f707da74e5f0. 2023-11-28 03:54:33,521 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for a075e19a5494ecfa5418f707da74e5f0: 2023-11-28 03:54:33,521 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3512): Not adding moved region record: a075e19a5494ecfa5418f707da74e5f0 to self. 2023-11-28 03:54:33,523 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed a075e19a5494ecfa5418f707da74e5f0 2023-11-28 03:54:33,523 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=51 updating hbase:meta row=a075e19a5494ecfa5418f707da74e5f0, regionState=CLOSED 2023-11-28 03:54:33,523 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701143657995.a075e19a5494ecfa5418f707da74e5f0.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1701143673523"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1701143673523"}]},"ts":"1701143673523"} 2023-11-28 03:54:33,526 INFO [PEWorker-1] procedure2.ProcedureExecutor(1825): Finished subprocedure pid=52, resume processing ppid=51 2023-11-28 03:54:33,526 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=52, ppid=51, state=SUCCESS; CloseRegionProcedure a075e19a5494ecfa5418f707da74e5f0, server=jenkins-hbase4.apache.org,41203,1701143600653 in 162 msec 2023-11-28 03:54:33,526 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=51, ppid=50, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=a075e19a5494ecfa5418f707da74e5f0, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,41203,1701143600653; forceNewPlan=false, retain=true 2023-11-28 03:54:33,677 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=51 updating hbase:meta row=a075e19a5494ecfa5418f707da74e5f0, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:33,677 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701143657995.a075e19a5494ecfa5418f707da74e5f0.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1701143673676"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1701143673676"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1701143673676"}]},"ts":"1701143673676"} 2023-11-28 03:54:33,678 INFO [PEWorker-5] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=53, ppid=51, state=RUNNABLE; OpenRegionProcedure a075e19a5494ecfa5418f707da74e5f0, server=jenkins-hbase4.apache.org,41203,1701143600653}] 2023-11-28 03:54:33,830 DEBUG [RSProcedureDispatcher-pool-1] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:33,833 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701143657995.a075e19a5494ecfa5418f707da74e5f0. 2023-11-28 03:54:33,833 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => a075e19a5494ecfa5418f707da74e5f0, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701143657995.a075e19a5494ecfa5418f707da74e5f0.', STARTKEY => '', ENDKEY => ''} 2023-11-28 03:54:33,833 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly a075e19a5494ecfa5418f707da74e5f0 2023-11-28 03:54:33,833 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701143657995.a075e19a5494ecfa5418f707da74e5f0.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-28 03:54:33,833 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for a075e19a5494ecfa5418f707da74e5f0 2023-11-28 03:54:33,833 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for a075e19a5494ecfa5418f707da74e5f0 2023-11-28 03:54:33,835 INFO [StoreOpener-a075e19a5494ecfa5418f707da74e5f0-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region a075e19a5494ecfa5418f707da74e5f0 2023-11-28 03:54:33,835 DEBUG [StoreOpener-a075e19a5494ecfa5418f707da74e5f0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a075e19a5494ecfa5418f707da74e5f0/family1 2023-11-28 03:54:33,836 DEBUG [StoreOpener-a075e19a5494ecfa5418f707da74e5f0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a075e19a5494ecfa5418f707da74e5f0/family1 2023-11-28 03:54:33,836 INFO [StoreOpener-a075e19a5494ecfa5418f707da74e5f0-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a075e19a5494ecfa5418f707da74e5f0 columnFamilyName family1 2023-11-28 03:54:33,841 DEBUG [StoreOpener-a075e19a5494ecfa5418f707da74e5f0-1] regionserver.HStore(539): loaded hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a075e19a5494ecfa5418f707da74e5f0/family1/bbfa095caba54308a337ffd047506f00 2023-11-28 03:54:33,844 DEBUG [StoreOpener-a075e19a5494ecfa5418f707da74e5f0-1] regionserver.HStore(539): loaded hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a075e19a5494ecfa5418f707da74e5f0/family1/ecdce964143d4d76961d70a907097fcb_SeqId_53_ 2023-11-28 03:54:33,844 INFO [StoreOpener-a075e19a5494ecfa5418f707da74e5f0-1] regionserver.HStore(310): Store=a075e19a5494ecfa5418f707da74e5f0/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-28 03:54:33,845 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a075e19a5494ecfa5418f707da74e5f0 2023-11-28 03:54:33,846 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a075e19a5494ecfa5418f707da74e5f0 2023-11-28 03:54:33,848 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for a075e19a5494ecfa5418f707da74e5f0 2023-11-28 03:54:33,849 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened a075e19a5494ecfa5418f707da74e5f0; next sequenceid=56; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74314413, jitterRate=0.10737104713916779}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-11-28 03:54:33,849 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for a075e19a5494ecfa5418f707da74e5f0: 2023-11-28 03:54:33,850 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2339): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701143657995.a075e19a5494ecfa5418f707da74e5f0., pid=53, masterSystemTime=1701143673830 2023-11-28 03:54:33,851 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2366): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701143657995.a075e19a5494ecfa5418f707da74e5f0. 2023-11-28 03:54:33,851 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701143657995.a075e19a5494ecfa5418f707da74e5f0. 2023-11-28 03:54:33,852 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=51 updating hbase:meta row=a075e19a5494ecfa5418f707da74e5f0, regionState=OPEN, openSeqNum=56, regionLocation=jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:33,852 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701143657995.a075e19a5494ecfa5418f707da74e5f0.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1701143673852"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1701143673852"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1701143673852"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1701143673852"}]},"ts":"1701143673852"} 2023-11-28 03:54:33,854 INFO [PEWorker-3] procedure2.ProcedureExecutor(1825): Finished subprocedure pid=53, resume processing ppid=51 2023-11-28 03:54:33,854 INFO [PEWorker-3] procedure2.ProcedureExecutor(1409): Finished pid=53, ppid=51, state=SUCCESS; OpenRegionProcedure a075e19a5494ecfa5418f707da74e5f0, server=jenkins-hbase4.apache.org,41203,1701143600653 in 175 msec 2023-11-28 03:54:33,856 INFO [PEWorker-1] procedure2.ProcedureExecutor(1825): Finished subprocedure pid=51, resume processing ppid=50 2023-11-28 03:54:33,856 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=51, ppid=50, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=a075e19a5494ecfa5418f707da74e5f0, REOPEN/MOVE in 495 msec 2023-11-28 03:54:33,858 INFO [PEWorker-2] procedure2.ProcedureExecutor(1825): Finished subprocedure pid=50, resume processing ppid=49 2023-11-28 03:54:33,858 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=50, ppid=49, state=SUCCESS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly in 501 msec 2023-11-28 03:54:33,860 INFO [PEWorker-4] procedure2.ProcedureExecutor(1409): Finished pid=49, state=SUCCESS; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly in 917 msec 2023-11-28 03:54:33,861 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(1230): Checking to see if procedure is done pid=49 2023-11-28 03:54:33,862 INFO [Listener at localhost/42853] compactions.TestMobCompactor(278): alter status finished 2023-11-28 03:54:33,863 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=38593] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-28 03:54:33,864 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] procedure2.ProcedureExecutor(1028): Stored pid=54, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-11-28 03:54:33,865 DEBUG [PEWorker-5] locking.LockProcedure(309): LOCKED pid=54, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-11-28 03:54:33,865 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-11-28 03:54:33,865 INFO [PEWorker-5] procedure2.TimeoutExecutorThread(81): ADDED pid=54, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1701144273865 2023-11-28 03:54:33,866 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-11-28 03:54:33,866 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 4 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column 'family1' 2023-11-28 03:54:33,866 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-11-28 03:54:33,868 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] zookeeper.ReadOnlyZKClient(139): Connect 0x5eea5707 to 127.0.0.1:55641 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-11-28 03:54:33,872 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] ipc.AbstractRpcClient(189): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@56449cc0, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-11-28 03:54:33,872 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-11-28 03:54:33,872 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-11-28 03:54:33,872 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-11-28 03:54:33,872 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-11-28 03:54:33,872 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-11-28 03:54:33,872 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-11-28 03:54:33,872 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-11-28 03:54:33,872 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-11-28 03:54:33,872 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-11-28 03:54:33,873 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-11-28 03:54:33,873 DEBUG [PEWorker-3] locking.LockProcedure(242): UNLOCKED pid=54, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-11-28 03:54:33,874 INFO [PEWorker-3] procedure2.ProcedureExecutor(1409): Finished pid=54, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE in 10 msec 2023-11-28 03:54:33,923 INFO [Listener at localhost/42853] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly Thread=248 (was 261), OpenFileDescriptor=652 (was 648) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=169 (was 196), ProcessCount=172 (was 172), AvailableMemoryMB=6964 (was 6977) 2023-11-28 03:54:33,935 INFO [Listener at localhost/42853] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily Thread=248, OpenFileDescriptor=652, MaxFileDescriptor=60000, SystemLoadAverage=169, ProcessCount=172, AvailableMemoryMB=6963 2023-11-28 03:54:33,936 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-11-28 03:54:33,938 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] procedure2.ProcedureExecutor(1028): Stored pid=55, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:33,939 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_PRE_OPERATION 2023-11-28 03:54:33,939 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(700): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily" procId is: 55 2023-11-28 03:54:33,940 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-11-28 03:54:33,940 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(1230): Checking to see if procedure is done pid=55 2023-11-28 03:54:33,941 DEBUG [HFileArchiver-10] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/99b453beb9cd8e4a604b7044c921115b 2023-11-28 03:54:33,941 DEBUG [HFileArchiver-10] backup.HFileArchiver(153): Directory hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/99b453beb9cd8e4a604b7044c921115b empty. 2023-11-28 03:54:33,942 DEBUG [HFileArchiver-10] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/99b453beb9cd8e4a604b7044c921115b 2023-11-28 03:54:33,942 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily regions 2023-11-28 03:54:34,041 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(1230): Checking to see if procedure is done pid=55 2023-11-28 03:54:34,242 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(1230): Checking to see if procedure is done pid=55 2023-11-28 03:54:34,351 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000001 2023-11-28 03:54:34,353 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(7675): creating {ENCODED => 99b453beb9cd8e4a604b7044c921115b, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701143673936.99b453beb9cd8e4a604b7044c921115b.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}, regionDir=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/.tmp 2023-11-28 03:54:34,543 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(1230): Checking to see if procedure is done pid=55 2023-11-28 03:54:34,759 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701143673936.99b453beb9cd8e4a604b7044c921115b.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-28 03:54:34,760 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1604): Closing 99b453beb9cd8e4a604b7044c921115b, disabling compactions & flushes 2023-11-28 03:54:34,760 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701143673936.99b453beb9cd8e4a604b7044c921115b. 2023-11-28 03:54:34,760 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701143673936.99b453beb9cd8e4a604b7044c921115b. 2023-11-28 03:54:34,760 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701143673936.99b453beb9cd8e4a604b7044c921115b. after waiting 0 ms 2023-11-28 03:54:34,760 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701143673936.99b453beb9cd8e4a604b7044c921115b. 2023-11-28 03:54:34,760 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701143673936.99b453beb9cd8e4a604b7044c921115b. 2023-11-28 03:54:34,760 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1558): Region close journal for 99b453beb9cd8e4a604b7044c921115b: 2023-11-28 03:54:34,762 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_ADD_TO_META 2023-11-28 03:54:34,763 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701143673936.99b453beb9cd8e4a604b7044c921115b.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1701143674763"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1701143674763"}]},"ts":"1701143674763"} 2023-11-28 03:54:34,764 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-11-28 03:54:34,765 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-11-28 03:54:34,765 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1701143674765"}]},"ts":"1701143674765"} 2023-11-28 03:54:34,766 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, state=ENABLING in hbase:meta 2023-11-28 03:54:34,769 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=56, ppid=55, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=99b453beb9cd8e4a604b7044c921115b, ASSIGN}] 2023-11-28 03:54:34,770 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=56, ppid=55, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=99b453beb9cd8e4a604b7044c921115b, ASSIGN 2023-11-28 03:54:34,771 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=56, ppid=55, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=99b453beb9cd8e4a604b7044c921115b, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,41203,1701143600653; forceNewPlan=false, retain=false 2023-11-28 03:54:34,922 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=56 updating hbase:meta row=99b453beb9cd8e4a604b7044c921115b, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:34,922 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701143673936.99b453beb9cd8e4a604b7044c921115b.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1701143674922"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1701143674922"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1701143674922"}]},"ts":"1701143674922"} 2023-11-28 03:54:34,924 INFO [PEWorker-4] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=57, ppid=56, state=RUNNABLE; OpenRegionProcedure 99b453beb9cd8e4a604b7044c921115b, server=jenkins-hbase4.apache.org,41203,1701143600653}] 2023-11-28 03:54:35,044 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(1230): Checking to see if procedure is done pid=55 2023-11-28 03:54:35,076 DEBUG [RSProcedureDispatcher-pool-2] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:35,079 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701143673936.99b453beb9cd8e4a604b7044c921115b. 2023-11-28 03:54:35,079 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 99b453beb9cd8e4a604b7044c921115b, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701143673936.99b453beb9cd8e4a604b7044c921115b.', STARTKEY => '', ENDKEY => ''} 2023-11-28 03:54:35,079 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 99b453beb9cd8e4a604b7044c921115b 2023-11-28 03:54:35,079 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701143673936.99b453beb9cd8e4a604b7044c921115b.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-28 03:54:35,079 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 99b453beb9cd8e4a604b7044c921115b 2023-11-28 03:54:35,079 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 99b453beb9cd8e4a604b7044c921115b 2023-11-28 03:54:35,081 INFO [StoreOpener-99b453beb9cd8e4a604b7044c921115b-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 99b453beb9cd8e4a604b7044c921115b 2023-11-28 03:54:35,082 DEBUG [StoreOpener-99b453beb9cd8e4a604b7044c921115b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/99b453beb9cd8e4a604b7044c921115b/family1 2023-11-28 03:54:35,082 DEBUG [StoreOpener-99b453beb9cd8e4a604b7044c921115b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/99b453beb9cd8e4a604b7044c921115b/family1 2023-11-28 03:54:35,082 INFO [StoreOpener-99b453beb9cd8e4a604b7044c921115b-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 99b453beb9cd8e4a604b7044c921115b columnFamilyName family1 2023-11-28 03:54:35,084 INFO [StoreOpener-99b453beb9cd8e4a604b7044c921115b-1] regionserver.HStore(310): Store=99b453beb9cd8e4a604b7044c921115b/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-28 03:54:35,085 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/99b453beb9cd8e4a604b7044c921115b 2023-11-28 03:54:35,085 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/99b453beb9cd8e4a604b7044c921115b 2023-11-28 03:54:35,088 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 99b453beb9cd8e4a604b7044c921115b 2023-11-28 03:54:35,090 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/99b453beb9cd8e4a604b7044c921115b/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-11-28 03:54:35,090 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 99b453beb9cd8e4a604b7044c921115b; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69702865, jitterRate=0.038653627038002014}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-11-28 03:54:35,090 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 99b453beb9cd8e4a604b7044c921115b: 2023-11-28 03:54:35,091 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2339): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701143673936.99b453beb9cd8e4a604b7044c921115b., pid=57, masterSystemTime=1701143675076 2023-11-28 03:54:35,092 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2366): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701143673936.99b453beb9cd8e4a604b7044c921115b. 2023-11-28 03:54:35,092 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701143673936.99b453beb9cd8e4a604b7044c921115b. 2023-11-28 03:54:35,093 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=56 updating hbase:meta row=99b453beb9cd8e4a604b7044c921115b, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:35,093 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701143673936.99b453beb9cd8e4a604b7044c921115b.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1701143675092"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1701143675092"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1701143675092"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1701143675092"}]},"ts":"1701143675092"} 2023-11-28 03:54:35,095 INFO [PEWorker-3] procedure2.ProcedureExecutor(1825): Finished subprocedure pid=57, resume processing ppid=56 2023-11-28 03:54:35,095 INFO [PEWorker-3] procedure2.ProcedureExecutor(1409): Finished pid=57, ppid=56, state=SUCCESS; OpenRegionProcedure 99b453beb9cd8e4a604b7044c921115b, server=jenkins-hbase4.apache.org,41203,1701143600653 in 170 msec 2023-11-28 03:54:35,097 INFO [PEWorker-1] procedure2.ProcedureExecutor(1825): Finished subprocedure pid=56, resume processing ppid=55 2023-11-28 03:54:35,097 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=56, ppid=55, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=99b453beb9cd8e4a604b7044c921115b, ASSIGN in 326 msec 2023-11-28 03:54:35,097 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-11-28 03:54:35,098 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1701143675097"}]},"ts":"1701143675097"} 2023-11-28 03:54:35,099 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, state=ENABLED in hbase:meta 2023-11-28 03:54:35,101 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_POST_OPERATION 2023-11-28 03:54:35,102 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=55, state=SUCCESS; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily in 1.1640 sec 2023-11-28 03:54:36,045 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(1230): Checking to see if procedure is done pid=55 2023-11-28 03:54:36,045 INFO [Listener at localhost/42853] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, procId: 55 completed 2023-11-28 03:54:36,068 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-28 03:54:36,069 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:36,070 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-11-28 03:54:36,070 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:54:36,070 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-11-28 03:54:36,070 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-11-28 03:54:36,070 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:36,070 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:36,072 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:54:36,072 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:36,072 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:54:36,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:54:36,073 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:36,073 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-28 03:54:36,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:36,073 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:36,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-28 03:54:36,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:36,074 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:36,074 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:36,074 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-11-28 03:54:36,074 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:54:36,074 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-11-28 03:54:36,075 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-28 03:54:36,075 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-28 03:54:36,075 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701143673936.99b453beb9cd8e4a604b7044c921115b. 2023-11-28 03:54:36,075 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701143673936.99b453beb9cd8e4a604b7044c921115b. started... 2023-11-28 03:54:36,076 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 99b453beb9cd8e4a604b7044c921115b 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-11-28 03:54:36,501 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511303c5045422efa4979805652ad718fa56f to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201511303c5045422efa4979805652ad718fa56f 2023-11-28 03:54:36,905 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/99b453beb9cd8e4a604b7044c921115b/.tmp/family1/87ce5a41564c45e788ba9ea6bc5ee862 2023-11-28 03:54:36,911 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/99b453beb9cd8e4a604b7044c921115b/.tmp/family1/87ce5a41564c45e788ba9ea6bc5ee862 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/99b453beb9cd8e4a604b7044c921115b/family1/87ce5a41564c45e788ba9ea6bc5ee862 2023-11-28 03:54:36,915 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/99b453beb9cd8e4a604b7044c921115b/family1/87ce5a41564c45e788ba9ea6bc5ee862, entries=1000, sequenceid=5, filesize=190.5 K 2023-11-28 03:54:36,915 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for 99b453beb9cd8e4a604b7044c921115b in 839ms, sequenceid=5, compaction requested=false 2023-11-28 03:54:36,915 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-11-28 03:54:36,916 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 99b453beb9cd8e4a604b7044c921115b: 2023-11-28 03:54:36,916 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701143673936.99b453beb9cd8e4a604b7044c921115b. 2023-11-28 03:54:36,916 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-28 03:54:36,916 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-28 03:54:36,916 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:36,916 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-11-28 03:54:36,916 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-11-28 03:54:36,918 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:36,918 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:36,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:36,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:54:36,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:54:36,918 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:36,918 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-28 03:54:36,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:54:36,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:54:36,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:36,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:36,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:54:36,920 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-11-28 03:54:36,920 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@594f24b2[Count = 0] remaining members to acquire global barrier 2023-11-28 03:54:36,920 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-11-28 03:54:36,920 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:36,921 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:36,921 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:36,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:36,921 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-11-28 03:54:36,921 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-11-28 03:54:36,921 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,41203,1701143600653' in zk 2023-11-28 03:54:36,921 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:36,921 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-28 03:54:36,922 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-11-28 03:54:36,922 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:36,922 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:54:36,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:36,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:54:36,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:54:36,923 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-11-28 03:54:36,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:54:36,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:54:36,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:36,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:36,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:54:36,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:36,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:36,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,41203,1701143600653': 2023-11-28 03:54:36,925 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-11-28 03:54:36,925 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-11-28 03:54:36,925 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-28 03:54:36,925 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-28 03:54:36,925 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:36,925 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-28 03:54:36,935 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:36,935 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:36,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:36,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:54:36,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:54:36,935 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:36,935 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:54:36,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:36,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:54:36,936 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:36,936 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:54:36,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:54:36,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:36,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:36,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:54:36,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:36,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:36,937 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:36,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:54:36,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:36,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:36,939 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:36,939 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:36,939 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:54:36,940 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:54:36,940 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-11-28 03:54:36,940 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-28 03:54:36,940 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:36,940 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:54:36,940 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:54:36,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:54:36,940 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:36,940 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-11-28 03:54:36,940 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-28 03:54:36,940 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:36,940 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:36,940 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:54:36,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:54:36,940 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:37,040 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-28 03:54:37,041 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-28 03:54:37,045 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-28 03:54:37,047 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-11-28 03:54:37,047 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:37,047 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-11-28 03:54:37,047 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:54:37,047 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-11-28 03:54:37,047 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-11-28 03:54:37,047 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:37,047 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:37,049 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:54:37,049 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:37,049 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:54:37,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:54:37,049 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:37,049 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-28 03:54:37,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:37,049 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:37,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-28 03:54:37,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:37,050 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:37,050 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-11-28 03:54:37,050 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:37,050 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-11-28 03:54:37,050 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:54:37,050 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-11-28 03:54:37,050 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-28 03:54:37,050 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-28 03:54:37,050 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701143673936.99b453beb9cd8e4a604b7044c921115b. 2023-11-28 03:54:37,051 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701143673936.99b453beb9cd8e4a604b7044c921115b. started... 2023-11-28 03:54:37,051 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 99b453beb9cd8e4a604b7044c921115b 1/1 column families, dataSize=74 B heapSize=408 B 2023-11-28 03:54:37,180 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-11-28 03:54:37,243 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x5eea5707 to 127.0.0.1:55641 2023-11-28 03:54:37,243 DEBUG [Finalizer] ipc.AbstractRpcClient(489): Stopping rpc client 2023-11-28 03:54:37,243 INFO [Finalizer] client.ConnectionImplementation(1973): Closing master protocol: MasterService 2023-11-28 03:54:37,243 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x0b70e2fa to 127.0.0.1:55641 2023-11-28 03:54:37,243 DEBUG [Finalizer] ipc.AbstractRpcClient(489): Stopping rpc client 2023-11-28 03:54:37,243 INFO [Finalizer] client.ConnectionImplementation(1973): Closing master protocol: MasterService 2023-11-28 03:54:37,243 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x4163f506 to 127.0.0.1:55641 2023-11-28 03:54:37,243 DEBUG [Finalizer] ipc.AbstractRpcClient(489): Stopping rpc client 2023-11-28 03:54:37,244 INFO [Finalizer] client.ConnectionImplementation(1973): Closing master protocol: MasterService 2023-11-28 03:54:37,244 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x0e5b8d27 to 127.0.0.1:55641 2023-11-28 03:54:37,244 DEBUG [Finalizer] ipc.AbstractRpcClient(489): Stopping rpc client 2023-11-28 03:54:37,244 INFO [Finalizer] client.ConnectionImplementation(1973): Closing master protocol: MasterService 2023-11-28 03:54:37,244 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x02451af3 to 127.0.0.1:55641 2023-11-28 03:54:37,244 DEBUG [Finalizer] ipc.AbstractRpcClient(489): Stopping rpc client 2023-11-28 03:54:37,463 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511283c07c5a0046b4bc683845aad7298321c to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201511283c07c5a0046b4bc683845aad7298321c 2023-11-28 03:54:37,869 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/99b453beb9cd8e4a604b7044c921115b/.tmp/family1/39e59533461c4a9abaa084c693cdb9f6 2023-11-28 03:54:37,873 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/99b453beb9cd8e4a604b7044c921115b/.tmp/family1/39e59533461c4a9abaa084c693cdb9f6 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/99b453beb9cd8e4a604b7044c921115b/family1/39e59533461c4a9abaa084c693cdb9f6 2023-11-28 03:54:37,877 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/99b453beb9cd8e4a604b7044c921115b/family1/39e59533461c4a9abaa084c693cdb9f6, entries=1, sequenceid=9, filesize=5.0 K 2023-11-28 03:54:37,877 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for 99b453beb9cd8e4a604b7044c921115b in 826ms, sequenceid=9, compaction requested=false 2023-11-28 03:54:37,878 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 99b453beb9cd8e4a604b7044c921115b: 2023-11-28 03:54:37,878 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701143673936.99b453beb9cd8e4a604b7044c921115b. 2023-11-28 03:54:37,878 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-28 03:54:37,878 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-28 03:54:37,878 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:37,878 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-11-28 03:54:37,878 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-11-28 03:54:37,880 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:37,880 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:37,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:37,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:54:37,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:54:37,880 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:37,880 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-28 03:54:37,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:54:37,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:54:37,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:37,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:37,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:54:37,882 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-11-28 03:54:37,882 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3005a167[Count = 0] remaining members to acquire global barrier 2023-11-28 03:54:37,882 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-11-28 03:54:37,882 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:37,883 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:37,883 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:37,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:37,883 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-11-28 03:54:37,883 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-11-28 03:54:37,883 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,41203,1701143600653' in zk 2023-11-28 03:54:37,883 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:37,883 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-28 03:54:37,885 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:37,885 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-11-28 03:54:37,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:37,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:54:37,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:54:37,885 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:54:37,886 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-11-28 03:54:37,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:54:37,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:54:37,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:37,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:37,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:54:37,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:37,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:37,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,41203,1701143600653': 2023-11-28 03:54:37,888 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-11-28 03:54:37,888 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-11-28 03:54:37,888 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-28 03:54:37,888 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-28 03:54:37,888 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:37,888 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-28 03:54:37,890 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:37,890 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:37,890 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:37,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:37,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:54:37,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:54:37,890 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:54:37,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:37,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:54:37,890 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:37,890 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:54:37,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:54:37,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:37,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:37,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:54:37,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:37,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:37,891 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:37,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:54:37,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:37,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:37,894 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:37,894 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:54:37,894 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:37,894 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:54:37,894 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-11-28 03:54:37,894 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-28 03:54:37,894 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:54:37,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:54:37,894 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:54:37,894 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:37,894 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:37,894 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:37,894 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-11-28 03:54:37,894 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-28 03:54:37,894 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:37,895 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:37,895 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:54:37,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:54:37,995 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-28 03:54:37,995 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-28 03:54:37,999 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-28 03:54:38,001 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-11-28 03:54:38,001 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:38,001 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-11-28 03:54:38,001 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:54:38,001 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-11-28 03:54:38,001 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-11-28 03:54:38,001 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:38,001 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:38,003 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:38,003 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:54:38,003 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:54:38,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:54:38,003 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:38,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:38,003 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-28 03:54:38,004 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:38,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-28 03:54:38,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:38,004 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:38,005 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-11-28 03:54:38,005 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:38,005 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-11-28 03:54:38,005 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:54:38,005 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-11-28 03:54:38,005 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-28 03:54:38,005 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-28 03:54:38,005 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701143673936.99b453beb9cd8e4a604b7044c921115b. 2023-11-28 03:54:38,005 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701143673936.99b453beb9cd8e4a604b7044c921115b. started... 2023-11-28 03:54:38,005 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 99b453beb9cd8e4a604b7044c921115b 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-28 03:54:38,424 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512010e622c27fb3a44b88523a6abe10940e0 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201512010e622c27fb3a44b88523a6abe10940e0 2023-11-28 03:54:38,830 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/99b453beb9cd8e4a604b7044c921115b/.tmp/family1/c222c6b504d843178d7372b76072fd12 2023-11-28 03:54:38,834 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/99b453beb9cd8e4a604b7044c921115b/.tmp/family1/c222c6b504d843178d7372b76072fd12 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/99b453beb9cd8e4a604b7044c921115b/family1/c222c6b504d843178d7372b76072fd12 2023-11-28 03:54:38,838 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/99b453beb9cd8e4a604b7044c921115b/family1/c222c6b504d843178d7372b76072fd12, entries=1, sequenceid=13, filesize=5.0 K 2023-11-28 03:54:38,839 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 99b453beb9cd8e4a604b7044c921115b in 833ms, sequenceid=13, compaction requested=true 2023-11-28 03:54:38,839 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 99b453beb9cd8e4a604b7044c921115b: 2023-11-28 03:54:38,839 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701143673936.99b453beb9cd8e4a604b7044c921115b. 2023-11-28 03:54:38,839 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-28 03:54:38,839 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-28 03:54:38,839 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:38,839 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-11-28 03:54:38,839 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-11-28 03:54:38,841 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:38,841 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:38,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:38,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:54:38,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:54:38,842 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:38,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:54:38,842 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-28 03:54:38,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:54:38,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:38,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:38,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:54:38,843 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-11-28 03:54:38,843 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@27b5aff2[Count = 0] remaining members to acquire global barrier 2023-11-28 03:54:38,843 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-11-28 03:54:38,843 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:38,845 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:38,845 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:38,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:38,845 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-11-28 03:54:38,845 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-11-28 03:54:38,845 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,41203,1701143600653' in zk 2023-11-28 03:54:38,845 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:38,845 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-28 03:54:38,846 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:38,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:38,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:54:38,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:54:38,846 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-11-28 03:54:38,846 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:54:38,846 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-11-28 03:54:38,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:54:38,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:54:38,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:38,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:38,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:54:38,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:38,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:38,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,41203,1701143600653': 2023-11-28 03:54:38,849 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-11-28 03:54:38,849 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-11-28 03:54:38,849 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-28 03:54:38,849 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-28 03:54:38,849 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:38,849 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-28 03:54:38,851 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:38,851 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:38,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:38,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:54:38,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:54:38,851 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:38,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:38,851 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:54:38,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:54:38,851 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:54:38,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:54:38,851 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:38,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:38,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:38,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:54:38,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:38,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:38,853 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:38,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:54:38,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:38,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:38,859 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:38,859 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:54:38,859 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:38,859 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:54:38,859 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:54:38,859 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:38,859 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:54:38,859 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:38,859 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-11-28 03:54:38,859 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-28 03:54:38,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:54:38,859 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:38,859 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:38,859 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:38,860 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-11-28 03:54:38,860 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-28 03:54:38,860 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:54:38,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:54:38,960 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-28 03:54:38,961 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-28 03:54:38,964 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-28 03:54:38,966 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-11-28 03:54:38,966 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:38,966 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-11-28 03:54:38,966 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:54:38,966 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-11-28 03:54:38,966 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-11-28 03:54:38,967 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:38,967 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:38,971 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:38,971 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:54:38,971 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:54:38,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:54:38,971 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:38,971 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-28 03:54:38,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:38,972 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:38,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-28 03:54:38,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:38,972 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:38,972 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-11-28 03:54:38,973 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:38,973 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-11-28 03:54:38,973 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:54:38,973 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-11-28 03:54:38,973 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-28 03:54:38,973 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-28 03:54:38,973 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701143673936.99b453beb9cd8e4a604b7044c921115b. 2023-11-28 03:54:38,973 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701143673936.99b453beb9cd8e4a604b7044c921115b. started... 2023-11-28 03:54:38,973 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 99b453beb9cd8e4a604b7044c921115b 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-28 03:54:39,386 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015120583ef133e9ae9427da9b26cc0443d5fe2 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2015120583ef133e9ae9427da9b26cc0443d5fe2 2023-11-28 03:54:39,794 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/99b453beb9cd8e4a604b7044c921115b/.tmp/family1/29b42335b68149bf8b264586e11860e6 2023-11-28 03:54:39,799 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/99b453beb9cd8e4a604b7044c921115b/.tmp/family1/29b42335b68149bf8b264586e11860e6 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/99b453beb9cd8e4a604b7044c921115b/family1/29b42335b68149bf8b264586e11860e6 2023-11-28 03:54:39,803 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/99b453beb9cd8e4a604b7044c921115b/family1/29b42335b68149bf8b264586e11860e6, entries=1, sequenceid=17, filesize=5.0 K 2023-11-28 03:54:39,804 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 99b453beb9cd8e4a604b7044c921115b in 831ms, sequenceid=17, compaction requested=true 2023-11-28 03:54:39,804 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 99b453beb9cd8e4a604b7044c921115b: 2023-11-28 03:54:39,804 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701143673936.99b453beb9cd8e4a604b7044c921115b. 2023-11-28 03:54:39,804 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-28 03:54:39,804 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-28 03:54:39,804 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:39,805 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-11-28 03:54:39,805 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-11-28 03:54:39,806 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:39,807 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:39,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:39,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:54:39,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:54:39,807 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:39,807 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-28 03:54:39,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:54:39,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:54:39,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:39,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:39,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:54:39,809 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-11-28 03:54:39,809 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7c4ae4e3[Count = 0] remaining members to acquire global barrier 2023-11-28 03:54:39,809 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-11-28 03:54:39,809 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:39,810 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:39,810 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:39,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:39,810 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-11-28 03:54:39,810 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-11-28 03:54:39,810 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,41203,1701143600653' in zk 2023-11-28 03:54:39,810 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:39,810 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-28 03:54:39,812 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:39,813 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-11-28 03:54:39,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:39,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:54:39,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:54:39,813 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:54:39,813 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-11-28 03:54:39,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:54:39,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:54:39,814 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:39,814 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:39,814 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:54:39,814 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:39,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:39,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,41203,1701143600653': 2023-11-28 03:54:39,815 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-11-28 03:54:39,815 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-11-28 03:54:39,815 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-28 03:54:39,815 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-28 03:54:39,815 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:39,815 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-28 03:54:39,817 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:39,817 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:39,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:39,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:54:39,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:54:39,817 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:39,817 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:54:39,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:39,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:54:39,818 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:39,818 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:54:39,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:54:39,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:39,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:39,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:54:39,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:39,819 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:39,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:39,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:54:39,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:39,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:39,822 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:54:39,822 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:54:39,822 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:54:39,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:54:39,822 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:54:39,822 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(399): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-11-28 03:54:39,822 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-11-28 03:54:39,822 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-28 03:54:39,822 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:54:39,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:54:39,822 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-11-28 03:54:39,822 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-28 03:54:39,826 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:39,826 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:39,827 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:39,827 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:39,827 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:39,827 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:39,827 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:39,923 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-28 03:54:39,924 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-28 03:54:39,927 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-28 03:54:39,929 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-11-28 03:54:39,929 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:39,929 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-11-28 03:54:39,929 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:54:39,929 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-11-28 03:54:39,929 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-11-28 03:54:39,930 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:39,930 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:39,931 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:39,931 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:54:39,931 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:54:39,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:54:39,931 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:39,931 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-28 03:54:39,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:39,932 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:39,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-28 03:54:39,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:39,932 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:39,932 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-11-28 03:54:39,933 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:39,933 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-11-28 03:54:39,933 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:54:39,933 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-11-28 03:54:39,933 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-28 03:54:39,933 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-28 03:54:39,933 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701143673936.99b453beb9cd8e4a604b7044c921115b. 2023-11-28 03:54:39,933 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701143673936.99b453beb9cd8e4a604b7044c921115b. started... 2023-11-28 03:54:39,933 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 99b453beb9cd8e4a604b7044c921115b 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-28 03:54:39,944 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015122801c58662869b4fbe86153ac1ce5fe357 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2015122801c58662869b4fbe86153ac1ce5fe357 2023-11-28 03:54:40,350 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/99b453beb9cd8e4a604b7044c921115b/.tmp/family1/a004d47b16bf4569885888eab5247319 2023-11-28 03:54:40,355 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/99b453beb9cd8e4a604b7044c921115b/.tmp/family1/a004d47b16bf4569885888eab5247319 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/99b453beb9cd8e4a604b7044c921115b/family1/a004d47b16bf4569885888eab5247319 2023-11-28 03:54:40,358 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/99b453beb9cd8e4a604b7044c921115b/family1/a004d47b16bf4569885888eab5247319, entries=1, sequenceid=21, filesize=5.0 K 2023-11-28 03:54:40,359 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 99b453beb9cd8e4a604b7044c921115b in 426ms, sequenceid=21, compaction requested=true 2023-11-28 03:54:40,359 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 99b453beb9cd8e4a604b7044c921115b: 2023-11-28 03:54:40,359 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701143673936.99b453beb9cd8e4a604b7044c921115b. 2023-11-28 03:54:40,359 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-28 03:54:40,359 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-28 03:54:40,359 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:40,359 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-11-28 03:54:40,359 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-11-28 03:54:40,364 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:40,364 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:40,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:40,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:54:40,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:54:40,364 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:40,364 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-28 03:54:40,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:54:40,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:54:40,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:40,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:40,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:54:40,366 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-11-28 03:54:40,366 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-11-28 03:54:40,366 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:40,366 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5c80256c[Count = 0] remaining members to acquire global barrier 2023-11-28 03:54:40,367 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:40,367 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:40,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:40,367 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-11-28 03:54:40,367 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-11-28 03:54:40,367 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,41203,1701143600653' in zk 2023-11-28 03:54:40,367 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:40,367 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-28 03:54:40,369 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-11-28 03:54:40,369 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:54:40,369 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:40,369 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-11-28 03:54:40,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:40,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:54:40,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:54:40,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:54:40,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:54:40,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:40,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:40,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:54:40,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:40,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:40,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,41203,1701143600653': 2023-11-28 03:54:40,371 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-11-28 03:54:40,371 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-11-28 03:54:40,371 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-28 03:54:40,371 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-28 03:54:40,371 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:40,371 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-28 03:54:40,373 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:40,373 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:40,373 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:40,373 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:54:40,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:40,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:40,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:54:40,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:54:40,373 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:54:40,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:54:40,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:54:40,373 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:40,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:40,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:40,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:54:40,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:40,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:40,375 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:40,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:54:40,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:40,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:40,378 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:40,378 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:40,378 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:54:40,378 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:40,378 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:54:40,378 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-11-28 03:54:40,378 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-28 03:54:40,378 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(399): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-11-28 03:54:40,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:54:40,378 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:54:40,378 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:40,378 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:54:40,378 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:40,378 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:40,379 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:40,379 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-11-28 03:54:40,379 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-28 03:54:40,379 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:54:40,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:54:40,479 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-28 03:54:40,480 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-28 03:54:40,483 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-28 03:54:40,485 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-11-28 03:54:40,485 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:40,485 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-11-28 03:54:40,485 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:54:40,485 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-11-28 03:54:40,485 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-11-28 03:54:40,486 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:40,486 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:40,487 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:40,487 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:54:40,487 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:54:40,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:54:40,487 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:40,487 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-28 03:54:40,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:40,488 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:40,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-28 03:54:40,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:40,488 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:40,489 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-11-28 03:54:40,489 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:40,489 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-11-28 03:54:40,489 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:54:40,489 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-11-28 03:54:40,489 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-28 03:54:40,489 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-28 03:54:40,489 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701143673936.99b453beb9cd8e4a604b7044c921115b. 2023-11-28 03:54:40,489 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701143673936.99b453beb9cd8e4a604b7044c921115b. started... 2023-11-28 03:54:40,489 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 99b453beb9cd8e4a604b7044c921115b 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-28 03:54:40,901 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151231c6f13448148f41979b1b8335035a7a55 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151231c6f13448148f41979b1b8335035a7a55 2023-11-28 03:54:41,307 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/99b453beb9cd8e4a604b7044c921115b/.tmp/family1/0bd66c99f1df4a31a3e23a20dbafb992 2023-11-28 03:54:41,312 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/99b453beb9cd8e4a604b7044c921115b/.tmp/family1/0bd66c99f1df4a31a3e23a20dbafb992 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/99b453beb9cd8e4a604b7044c921115b/family1/0bd66c99f1df4a31a3e23a20dbafb992 2023-11-28 03:54:41,315 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/99b453beb9cd8e4a604b7044c921115b/family1/0bd66c99f1df4a31a3e23a20dbafb992, entries=1, sequenceid=25, filesize=5.0 K 2023-11-28 03:54:41,316 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 99b453beb9cd8e4a604b7044c921115b in 827ms, sequenceid=25, compaction requested=true 2023-11-28 03:54:41,316 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 99b453beb9cd8e4a604b7044c921115b: 2023-11-28 03:54:41,316 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701143673936.99b453beb9cd8e4a604b7044c921115b. 2023-11-28 03:54:41,316 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-28 03:54:41,316 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-28 03:54:41,316 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:41,317 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-11-28 03:54:41,317 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-11-28 03:54:41,318 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:41,318 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:41,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:41,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:54:41,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:54:41,319 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:41,319 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-28 03:54:41,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:54:41,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:54:41,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:41,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:41,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:54:41,320 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-11-28 03:54:41,320 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4193b902[Count = 0] remaining members to acquire global barrier 2023-11-28 03:54:41,320 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-11-28 03:54:41,320 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:41,322 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:41,322 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:41,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:41,322 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-11-28 03:54:41,322 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-11-28 03:54:41,322 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,41203,1701143600653' in zk 2023-11-28 03:54:41,322 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:41,322 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-28 03:54:41,323 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:41,323 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-11-28 03:54:41,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:41,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:54:41,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:54:41,323 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:54:41,324 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-11-28 03:54:41,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:54:41,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:54:41,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:41,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:41,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:54:41,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:41,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:41,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,41203,1701143600653': 2023-11-28 03:54:41,326 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-11-28 03:54:41,326 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-11-28 03:54:41,326 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-28 03:54:41,326 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-28 03:54:41,326 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:41,326 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-28 03:54:41,327 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:41,327 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:41,327 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:41,327 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:54:41,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:41,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:41,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:54:41,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:54:41,328 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:54:41,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:54:41,328 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:41,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:54:41,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:41,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:41,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:54:41,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:41,329 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:41,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:41,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:54:41,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:41,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:41,331 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:41,332 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:41,332 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:41,332 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:41,332 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:54:41,332 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-11-28 03:54:41,332 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-28 03:54:41,332 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:54:41,332 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(399): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-11-28 03:54:41,332 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:54:41,332 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:41,332 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:54:41,332 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:41,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:54:41,332 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:41,332 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-11-28 03:54:41,332 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-28 03:54:41,332 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:54:41,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:54:41,432 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-28 03:54:41,433 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-28 03:54:41,437 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-28 03:54:41,438 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-11-28 03:54:41,438 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:41,438 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-11-28 03:54:41,438 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:54:41,438 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-11-28 03:54:41,438 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-11-28 03:54:41,439 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:41,439 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:41,443 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:41,443 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:54:41,443 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:54:41,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:54:41,443 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:41,443 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-28 03:54:41,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:41,444 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:41,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-28 03:54:41,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:41,444 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:41,448 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-11-28 03:54:41,448 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:41,448 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-11-28 03:54:41,448 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:54:41,448 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-11-28 03:54:41,448 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-28 03:54:41,448 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-28 03:54:41,448 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701143673936.99b453beb9cd8e4a604b7044c921115b. 2023-11-28 03:54:41,449 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701143673936.99b453beb9cd8e4a604b7044c921115b. started... 2023-11-28 03:54:41,449 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 99b453beb9cd8e4a604b7044c921115b 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-28 03:54:41,865 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601019048140fda264662b10180d436d216c1 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201601019048140fda264662b10180d436d216c1 2023-11-28 03:54:42,271 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/99b453beb9cd8e4a604b7044c921115b/.tmp/family1/47986f152f544c6a9851924a991b88cc 2023-11-28 03:54:42,276 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/99b453beb9cd8e4a604b7044c921115b/.tmp/family1/47986f152f544c6a9851924a991b88cc as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/99b453beb9cd8e4a604b7044c921115b/family1/47986f152f544c6a9851924a991b88cc 2023-11-28 03:54:42,280 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/99b453beb9cd8e4a604b7044c921115b/family1/47986f152f544c6a9851924a991b88cc, entries=1, sequenceid=29, filesize=5.0 K 2023-11-28 03:54:42,281 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 99b453beb9cd8e4a604b7044c921115b in 832ms, sequenceid=29, compaction requested=true 2023-11-28 03:54:42,281 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 99b453beb9cd8e4a604b7044c921115b: 2023-11-28 03:54:42,281 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701143673936.99b453beb9cd8e4a604b7044c921115b. 2023-11-28 03:54:42,281 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-28 03:54:42,281 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-28 03:54:42,281 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:42,281 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-11-28 03:54:42,281 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-11-28 03:54:42,283 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:42,283 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:42,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:42,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:54:42,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:54:42,283 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:42,283 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-28 03:54:42,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:54:42,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:54:42,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:42,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:42,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:54:42,285 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-11-28 03:54:42,285 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4b5344fd[Count = 0] remaining members to acquire global barrier 2023-11-28 03:54:42,285 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-11-28 03:54:42,285 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:42,286 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:42,286 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:42,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:42,286 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-11-28 03:54:42,286 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-11-28 03:54:42,286 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,41203,1701143600653' in zk 2023-11-28 03:54:42,286 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:42,286 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-28 03:54:42,288 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:42,288 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-11-28 03:54:42,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:42,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:54:42,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:54:42,288 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:54:42,288 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-11-28 03:54:42,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:54:42,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:54:42,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:42,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:42,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:54:42,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:42,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:42,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,41203,1701143600653': 2023-11-28 03:54:42,290 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-11-28 03:54:42,290 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-28 03:54:42,290 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-28 03:54:42,290 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-11-28 03:54:42,290 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:42,290 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-28 03:54:42,294 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:42,294 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:42,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:42,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:54:42,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:54:42,294 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:42,294 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:54:42,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:42,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:54:42,294 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:54:42,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:54:42,294 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:42,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:42,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:42,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:54:42,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:42,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:42,296 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:42,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:54:42,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:42,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:42,299 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:42,299 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:42,299 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:42,299 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:54:42,299 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:54:42,299 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:42,299 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-11-28 03:54:42,299 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-28 03:54:42,299 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:54:42,299 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:54:42,299 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:42,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:54:42,299 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:42,299 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:42,299 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-11-28 03:54:42,299 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-28 03:54:42,299 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:54:42,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:54:42,399 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-28 03:54:42,400 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-28 03:54:42,404 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-28 03:54:42,406 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-11-28 03:54:42,406 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:42,406 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-11-28 03:54:42,406 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:54:42,406 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-11-28 03:54:42,406 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-11-28 03:54:42,407 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:42,407 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:42,408 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:42,408 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:54:42,408 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:54:42,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:54:42,409 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:42,409 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-28 03:54:42,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:42,409 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:42,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-28 03:54:42,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:42,410 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:42,410 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-11-28 03:54:42,410 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:42,410 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-11-28 03:54:42,410 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:54:42,410 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-11-28 03:54:42,416 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-28 03:54:42,416 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-28 03:54:42,416 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701143673936.99b453beb9cd8e4a604b7044c921115b. 2023-11-28 03:54:42,416 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701143673936.99b453beb9cd8e4a604b7044c921115b. started... 2023-11-28 03:54:42,416 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 99b453beb9cd8e4a604b7044c921115b 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-28 03:54:42,829 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2016010367b6ab939cd04e6296c859adc36bba43 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2016010367b6ab939cd04e6296c859adc36bba43 2023-11-28 03:54:43,236 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/99b453beb9cd8e4a604b7044c921115b/.tmp/family1/67614a2883774e5d8d6500a2da56638a 2023-11-28 03:54:43,240 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/99b453beb9cd8e4a604b7044c921115b/.tmp/family1/67614a2883774e5d8d6500a2da56638a as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/99b453beb9cd8e4a604b7044c921115b/family1/67614a2883774e5d8d6500a2da56638a 2023-11-28 03:54:43,244 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/99b453beb9cd8e4a604b7044c921115b/family1/67614a2883774e5d8d6500a2da56638a, entries=1, sequenceid=33, filesize=5.0 K 2023-11-28 03:54:43,245 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 99b453beb9cd8e4a604b7044c921115b in 829ms, sequenceid=33, compaction requested=true 2023-11-28 03:54:43,245 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 99b453beb9cd8e4a604b7044c921115b: 2023-11-28 03:54:43,245 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701143673936.99b453beb9cd8e4a604b7044c921115b. 2023-11-28 03:54:43,245 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-28 03:54:43,245 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-28 03:54:43,245 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:43,245 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-11-28 03:54:43,245 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-11-28 03:54:43,246 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:43,246 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:43,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:43,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:54:43,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:54:43,247 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:43,247 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-28 03:54:43,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:54:43,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:54:43,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:43,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:43,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:54:43,248 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-11-28 03:54:43,248 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6095dc88[Count = 0] remaining members to acquire global barrier 2023-11-28 03:54:43,248 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-11-28 03:54:43,248 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:43,253 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:43,253 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:43,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:43,253 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-11-28 03:54:43,253 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-11-28 03:54:43,253 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,41203,1701143600653' in zk 2023-11-28 03:54:43,253 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:43,253 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-28 03:54:43,254 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-11-28 03:54:43,255 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:43,255 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:54:43,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:43,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:54:43,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:54:43,255 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-11-28 03:54:43,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:54:43,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:54:43,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:43,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:43,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:54:43,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:43,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:43,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,41203,1701143600653': 2023-11-28 03:54:43,257 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-11-28 03:54:43,257 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-11-28 03:54:43,257 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-28 03:54:43,257 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-28 03:54:43,257 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:43,257 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-28 03:54:43,259 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:43,259 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:43,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:43,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:54:43,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:54:43,259 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:43,259 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:54:43,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:43,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:54:43,259 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:43,259 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:54:43,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:54:43,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:43,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:43,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:54:43,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:43,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:43,260 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:43,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:54:43,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:43,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:43,264 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:43,264 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:43,264 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:54:43,264 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:54:43,264 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:54:43,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:54:43,264 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-11-28 03:54:43,264 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-28 03:54:43,264 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:54:43,264 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:43,264 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:43,264 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:43,264 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:54:43,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:54:43,264 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:43,264 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-11-28 03:54:43,265 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-28 03:54:43,264 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:43,365 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-28 03:54:43,366 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-28 03:54:43,370 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-28 03:54:43,371 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-11-28 03:54:43,371 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:43,371 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-11-28 03:54:43,371 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:54:43,371 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-11-28 03:54:43,372 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-11-28 03:54:43,372 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:43,372 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:43,373 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:43,373 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:54:43,373 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:54:43,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:54:43,374 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:43,374 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-28 03:54:43,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:43,374 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:43,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-28 03:54:43,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:43,374 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:43,375 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-11-28 03:54:43,375 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:43,375 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-11-28 03:54:43,375 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:54:43,375 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-11-28 03:54:43,376 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-28 03:54:43,376 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-28 03:54:43,376 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701143673936.99b453beb9cd8e4a604b7044c921115b. 2023-11-28 03:54:43,376 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701143673936.99b453beb9cd8e4a604b7044c921115b. started... 2023-11-28 03:54:43,376 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 99b453beb9cd8e4a604b7044c921115b 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-28 03:54:43,794 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201509072db958f78c8f48fbb54b51de4b9327da to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201509072db958f78c8f48fbb54b51de4b9327da 2023-11-28 03:54:43,799 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/99b453beb9cd8e4a604b7044c921115b/.tmp/family1/aa9a6fe34de641d08f12a1d8bb92c3d1 2023-11-28 03:54:43,803 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/99b453beb9cd8e4a604b7044c921115b/.tmp/family1/aa9a6fe34de641d08f12a1d8bb92c3d1 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/99b453beb9cd8e4a604b7044c921115b/family1/aa9a6fe34de641d08f12a1d8bb92c3d1 2023-11-28 03:54:43,809 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/99b453beb9cd8e4a604b7044c921115b/family1/aa9a6fe34de641d08f12a1d8bb92c3d1, entries=1, sequenceid=37, filesize=5.0 K 2023-11-28 03:54:43,810 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 99b453beb9cd8e4a604b7044c921115b in 434ms, sequenceid=37, compaction requested=true 2023-11-28 03:54:43,810 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 99b453beb9cd8e4a604b7044c921115b: 2023-11-28 03:54:43,810 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701143673936.99b453beb9cd8e4a604b7044c921115b. 2023-11-28 03:54:43,810 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-28 03:54:43,810 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-28 03:54:43,810 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:43,810 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-11-28 03:54:43,810 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-11-28 03:54:43,812 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:43,812 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:43,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:43,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:54:43,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:54:43,812 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:43,813 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-28 03:54:43,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:54:43,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:54:43,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:43,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:43,814 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:54:43,814 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-11-28 03:54:43,814 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3d632eb3[Count = 0] remaining members to acquire global barrier 2023-11-28 03:54:43,814 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-11-28 03:54:43,814 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:43,816 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:43,816 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:43,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:43,816 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-11-28 03:54:43,816 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:43,816 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-28 03:54:43,816 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-11-28 03:54:43,816 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,41203,1701143600653' in zk 2023-11-28 03:54:43,819 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-11-28 03:54:43,819 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:54:43,819 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:43,819 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-11-28 03:54:43,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:43,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:54:43,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:54:43,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:54:43,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:54:43,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:43,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:43,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:54:43,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:43,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:43,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,41203,1701143600653': 2023-11-28 03:54:43,821 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-11-28 03:54:43,821 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-11-28 03:54:43,821 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-28 03:54:43,821 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-28 03:54:43,821 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:43,821 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-28 03:54:43,823 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:43,823 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:43,823 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:43,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:43,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:54:43,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:54:43,823 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:54:43,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:43,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:54:43,824 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:54:43,824 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:43,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:54:43,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:43,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:43,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:54:43,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:43,825 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:43,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:43,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:54:43,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:43,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:43,827 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:43,827 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:54:43,827 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:43,827 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:54:43,827 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:54:43,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:54:43,827 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:54:43,827 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(399): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-11-28 03:54:43,827 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:43,827 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-11-28 03:54:43,827 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-28 03:54:43,827 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:54:43,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:54:43,827 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:43,828 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:43,828 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:43,828 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:43,828 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-11-28 03:54:43,828 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-28 03:54:43,928 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-28 03:54:43,929 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-28 03:54:43,933 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-28 03:54:43,934 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-11-28 03:54:43,934 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:43,934 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-11-28 03:54:43,934 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:54:43,934 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-11-28 03:54:43,934 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-11-28 03:54:43,935 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:43,935 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:43,936 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:43,936 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:54:43,936 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:54:43,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:54:43,937 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:43,937 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-28 03:54:43,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:43,937 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:43,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-28 03:54:43,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:43,937 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:43,938 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-11-28 03:54:43,938 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:43,938 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-11-28 03:54:43,938 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:54:43,938 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-11-28 03:54:43,938 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-28 03:54:43,938 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-28 03:54:43,938 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701143673936.99b453beb9cd8e4a604b7044c921115b. 2023-11-28 03:54:43,938 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701143673936.99b453beb9cd8e4a604b7044c921115b. started... 2023-11-28 03:54:43,938 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 99b453beb9cd8e4a604b7044c921115b 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-28 03:54:44,350 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511209be19f54a2bd451db9c2f3b2fffa42b0 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201511209be19f54a2bd451db9c2f3b2fffa42b0 2023-11-28 03:54:44,755 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/99b453beb9cd8e4a604b7044c921115b/.tmp/family1/02380d42bbf14106ae64f93c2bb53754 2023-11-28 03:54:44,760 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/99b453beb9cd8e4a604b7044c921115b/.tmp/family1/02380d42bbf14106ae64f93c2bb53754 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/99b453beb9cd8e4a604b7044c921115b/family1/02380d42bbf14106ae64f93c2bb53754 2023-11-28 03:54:44,763 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/99b453beb9cd8e4a604b7044c921115b/family1/02380d42bbf14106ae64f93c2bb53754, entries=1, sequenceid=41, filesize=5.0 K 2023-11-28 03:54:44,764 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 99b453beb9cd8e4a604b7044c921115b in 826ms, sequenceid=41, compaction requested=true 2023-11-28 03:54:44,765 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 99b453beb9cd8e4a604b7044c921115b: 2023-11-28 03:54:44,765 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701143673936.99b453beb9cd8e4a604b7044c921115b. 2023-11-28 03:54:44,765 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-28 03:54:44,765 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-28 03:54:44,765 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:44,765 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-11-28 03:54:44,765 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-11-28 03:54:44,767 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:44,767 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:44,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:44,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:54:44,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:54:44,768 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:44,768 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-28 03:54:44,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:54:44,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:54:44,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:44,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:44,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:54:44,769 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-11-28 03:54:44,769 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@61e4cbb4[Count = 0] remaining members to acquire global barrier 2023-11-28 03:54:44,769 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-11-28 03:54:44,769 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:44,770 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:44,770 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:44,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:44,770 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-11-28 03:54:44,770 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-11-28 03:54:44,770 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,41203,1701143600653' in zk 2023-11-28 03:54:44,770 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:44,771 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-28 03:54:44,772 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:44,772 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-11-28 03:54:44,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:44,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:54:44,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:54:44,772 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:54:44,772 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-11-28 03:54:44,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:54:44,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:54:44,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:44,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:44,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:54:44,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:44,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:44,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,41203,1701143600653': 2023-11-28 03:54:44,774 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-11-28 03:54:44,774 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-11-28 03:54:44,774 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-28 03:54:44,774 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-28 03:54:44,774 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:44,774 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-28 03:54:44,779 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:44,779 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:44,779 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:54:44,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:44,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:54:44,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:54:44,779 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:44,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:44,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:54:44,779 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:54:44,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:54:44,779 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:44,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:44,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:44,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:54:44,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:44,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:44,781 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:44,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:54:44,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:44,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:44,783 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:44,783 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:54:44,783 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:44,783 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:44,783 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-11-28 03:54:44,783 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-28 03:54:44,783 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:54:44,783 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:54:44,783 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(399): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-11-28 03:54:44,783 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:44,783 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-11-28 03:54:44,783 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-28 03:54:44,783 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:54:44,783 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:44,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:54:44,784 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:44,784 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:44,784 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:54:44,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:54:44,883 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-28 03:54:44,884 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-28 03:54:44,885 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=38593] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:44,887 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] procedure2.ProcedureExecutor(1028): Stored pid=58, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-11-28 03:54:44,887 DEBUG [PEWorker-4] locking.LockProcedure(309): LOCKED pid=58, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-11-28 03:54:44,888 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-11-28 03:54:44,888 INFO [PEWorker-4] procedure2.TimeoutExecutorThread(81): ADDED pid=58, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1701144284888 2023-11-28 03:54:44,890 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-11-28 03:54:44,891 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column 'family1' 2023-11-28 03:54:44,891 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-11-28 03:54:44,893 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] zookeeper.ReadOnlyZKClient(139): Connect 0x21180dfe to 127.0.0.1:55641 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-11-28 03:54:44,897 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] ipc.AbstractRpcClient(189): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@2da85145, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-11-28 03:54:44,897 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-11-28 03:54:45,360 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103bc0a028efc464b8aae424cfae633c74e to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20160103bc0a028efc464b8aae424cfae633c74e 2023-11-28 03:54:45,361 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-11-28 03:54:45,374 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(122): Using SIMPLE authentication for service=ClientService, sasl=false 2023-11-28 03:54:45,375 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:42522, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-11-28 03:54:45,383 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(122): Using SIMPLE authentication for service=MasterService, sasl=false 2023-11-28 03:54:45,384 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:41960, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-11-28 03:54:45,392 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151228/family1/dce341e4c35f406e9ef222bfcc50eea5 first=Optional[r3] last=Optional[r6] 2023-11-28 03:54:45,392 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701143673936.99b453beb9cd8e4a604b7044c921115b., hostname=jenkins-hbase4.apache.org,41203,1701143600653, seqNum=2 for row with hfile group [{family1,hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151228/family1/dce341e4c35f406e9ef222bfcc50eea5}] 2023-11-28 03:54:45,396 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HStore(763): Validating hfile at hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151228/family1/dce341e4c35f406e9ef222bfcc50eea5 for inclusion in 99b453beb9cd8e4a604b7044c921115b/family1 2023-11-28 03:54:45,399 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HStore(775): HFile bounds: first=r3 last=r6 2023-11-28 03:54:45,399 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HStore(777): Region bounds: first= last= 2023-11-28 03:54:45,399 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HRegion(2446): Flush status journal for 99b453beb9cd8e4a604b7044c921115b: 2023-11-28 03:54:45,400 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151228/family1/dce341e4c35f406e9ef222bfcc50eea5 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__rs70su8edd3nv27dmf7qnelaqfsv6crsflg6125ubvnpmmdv34hm32jvofips916/family1/dce341e4c35f406e9ef222bfcc50eea5 2023-11-28 03:54:45,402 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__rs70su8edd3nv27dmf7qnelaqfsv6crsflg6125ubvnpmmdv34hm32jvofips916/family1/dce341e4c35f406e9ef222bfcc50eea5 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/99b453beb9cd8e4a604b7044c921115b/family1/c2881aa614184c9bb062c7b74f497cc5_SeqId_44_ 2023-11-28 03:54:45,402 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HStore(849): Loaded HFile hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__rs70su8edd3nv27dmf7qnelaqfsv6crsflg6125ubvnpmmdv34hm32jvofips916/family1/dce341e4c35f406e9ef222bfcc50eea5 into 99b453beb9cd8e4a604b7044c921115b/family1 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/99b453beb9cd8e4a604b7044c921115b/family1/c2881aa614184c9bb062c7b74f497cc5_SeqId_44_ - updating store file list. 2023-11-28 03:54:45,406 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HStore(882): Loaded HFile hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/99b453beb9cd8e4a604b7044c921115b/family1/c2881aa614184c9bb062c7b74f497cc5_SeqId_44_ into 99b453beb9cd8e4a604b7044c921115b/family1 2023-11-28 03:54:45,406 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HStore(855): Successfully loaded hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__rs70su8edd3nv27dmf7qnelaqfsv6crsflg6125ubvnpmmdv34hm32jvofips916/family1/dce341e4c35f406e9ef222bfcc50eea5 into 99b453beb9cd8e4a604b7044c921115b/family1 (new location: hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/99b453beb9cd8e4a604b7044c921115b/family1/c2881aa614184c9bb062c7b74f497cc5_SeqId_44_) 2023-11-28 03:54:45,407 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__rs70su8edd3nv27dmf7qnelaqfsv6crsflg6125ubvnpmmdv34hm32jvofips916/family1/dce341e4c35f406e9ef222bfcc50eea5 2023-11-28 03:54:45,407 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-11-28 03:54:45,410 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 51552 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-11-28 03:54:45,410 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HStore(1912): 99b453beb9cd8e4a604b7044c921115b/family1 is initiating minor compaction 2023-11-28 03:54:45,410 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@5d008dcf; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-28 03:54:45,410 INFO [RS:0;jenkins-hbase4:41203-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 99b453beb9cd8e4a604b7044c921115b/family1 in testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701143673936.99b453beb9cd8e4a604b7044c921115b. 2023-11-28 03:54:45,411 INFO [RS:0;jenkins-hbase4:41203-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/99b453beb9cd8e4a604b7044c921115b/family1/39e59533461c4a9abaa084c693cdb9f6, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/99b453beb9cd8e4a604b7044c921115b/family1/c222c6b504d843178d7372b76072fd12, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/99b453beb9cd8e4a604b7044c921115b/family1/29b42335b68149bf8b264586e11860e6, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/99b453beb9cd8e4a604b7044c921115b/family1/a004d47b16bf4569885888eab5247319, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/99b453beb9cd8e4a604b7044c921115b/family1/0bd66c99f1df4a31a3e23a20dbafb992, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/99b453beb9cd8e4a604b7044c921115b/family1/47986f152f544c6a9851924a991b88cc, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/99b453beb9cd8e4a604b7044c921115b/family1/67614a2883774e5d8d6500a2da56638a, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/99b453beb9cd8e4a604b7044c921115b/family1/aa9a6fe34de641d08f12a1d8bb92c3d1, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/99b453beb9cd8e4a604b7044c921115b/family1/02380d42bbf14106ae64f93c2bb53754, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/99b453beb9cd8e4a604b7044c921115b/family1/c2881aa614184c9bb062c7b74f497cc5_SeqId_44_] into tmpdir=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/99b453beb9cd8e4a604b7044c921115b/.tmp, totalSize=50.3 K 2023-11-28 03:54:45,411 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] compactions.Compactor(207): Compacting 39e59533461c4a9abaa084c693cdb9f6, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-11-28 03:54:45,411 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] compactions.Compactor(207): Compacting c222c6b504d843178d7372b76072fd12, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=13 2023-11-28 03:54:45,411 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] compactions.Compactor(207): Compacting 29b42335b68149bf8b264586e11860e6, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=17 2023-11-28 03:54:45,411 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] compactions.Compactor(207): Compacting a004d47b16bf4569885888eab5247319, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=21 2023-11-28 03:54:45,411 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] compactions.Compactor(207): Compacting 0bd66c99f1df4a31a3e23a20dbafb992, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=25 2023-11-28 03:54:45,411 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] compactions.Compactor(207): Compacting 47986f152f544c6a9851924a991b88cc, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=29 2023-11-28 03:54:45,411 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] compactions.Compactor(207): Compacting 67614a2883774e5d8d6500a2da56638a, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33 2023-11-28 03:54:45,411 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] compactions.Compactor(207): Compacting aa9a6fe34de641d08f12a1d8bb92c3d1, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=37 2023-11-28 03:54:45,411 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] compactions.Compactor(207): Compacting 02380d42bbf14106ae64f93c2bb53754, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=41 2023-11-28 03:54:45,411 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] compactions.Compactor(207): Compacting c2881aa614184c9bb062c7b74f497cc5_SeqId_44_, keycount=4, bloomtype=ROW, size=5.6 K, encoding=NONE, compression=NONE, seqNum=45 2023-11-28 03:54:45,412 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-11-28 03:54:45,413 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-28 03:54:45,414 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2015122801c58662869b4fbe86153ac1ce5fe357 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2015122801c58662869b4fbe86153ac1ce5fe357 2023-11-28 03:54:45,416 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151231c6f13448148f41979b1b8335035a7a55 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151231c6f13448148f41979b1b8335035a7a55 2023-11-28 03:54:45,417 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201601019048140fda264662b10180d436d216c1 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201601019048140fda264662b10180d436d216c1 2023-11-28 03:54:45,418 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2016010367b6ab939cd04e6296c859adc36bba43 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2016010367b6ab939cd04e6296c859adc36bba43 2023-11-28 03:54:45,418 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 4 expired mob files are deleted 2023-11-28 03:54:45,418 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151228' 2023-11-28 03:54:45,418 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-11-28 03:54:45,418 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-11-28 03:54:45,418 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-11-28 03:54:45,418 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-11-28 03:54:45,418 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-11-28 03:54:45,418 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-11-28 03:54:45,418 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-11-28 03:54:45,418 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-11-28 03:54:45,449 INFO [RS:0;jenkins-hbase4:41203-shortCompactions-0] throttle.PressureAwareThroughputController(145): 99b453beb9cd8e4a604b7044c921115b#family1#compaction#72 average throughput is 0.79 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-11-28 03:54:45,875 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015120533108424ae324359b55340b5a0d95145 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2015120533108424ae324359b55340b5a0d95145 2023-11-28 03:54:45,876 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-11-28 03:54:45,908 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151130/family1/b2e227eea609419b972aab74f6937cb0 first=Optional[r00] last=Optional[r2] 2023-11-28 03:54:45,908 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701143673936.99b453beb9cd8e4a604b7044c921115b., hostname=jenkins-hbase4.apache.org,41203,1701143600653, seqNum=2 for row with hfile group [{family1,hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151130/family1/b2e227eea609419b972aab74f6937cb0}] 2023-11-28 03:54:45,910 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HStore(763): Validating hfile at hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151130/family1/b2e227eea609419b972aab74f6937cb0 for inclusion in 99b453beb9cd8e4a604b7044c921115b/family1 2023-11-28 03:54:45,913 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HStore(775): HFile bounds: first=r00 last=r2 2023-11-28 03:54:45,913 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HStore(777): Region bounds: first= last= 2023-11-28 03:54:45,913 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HRegion(2446): Flush status journal for 99b453beb9cd8e4a604b7044c921115b: 2023-11-28 03:54:45,914 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151130/family1/b2e227eea609419b972aab74f6937cb0 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__ughu86451cu6b4p8ipn8uj3th5jmo6u93p19j7mfm5ive9t58r24ouh8ic0h095u/family1/b2e227eea609419b972aab74f6937cb0 2023-11-28 03:54:45,916 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__ughu86451cu6b4p8ipn8uj3th5jmo6u93p19j7mfm5ive9t58r24ouh8ic0h095u/family1/b2e227eea609419b972aab74f6937cb0 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/99b453beb9cd8e4a604b7044c921115b/family1/2c79e042b8c642dd81727fb552b3c45c_SeqId_46_ 2023-11-28 03:54:45,917 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HStore(849): Loaded HFile hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__ughu86451cu6b4p8ipn8uj3th5jmo6u93p19j7mfm5ive9t58r24ouh8ic0h095u/family1/b2e227eea609419b972aab74f6937cb0 into 99b453beb9cd8e4a604b7044c921115b/family1 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/99b453beb9cd8e4a604b7044c921115b/family1/2c79e042b8c642dd81727fb552b3c45c_SeqId_46_ - updating store file list. 2023-11-28 03:54:45,920 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HStore(882): Loaded HFile hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/99b453beb9cd8e4a604b7044c921115b/family1/2c79e042b8c642dd81727fb552b3c45c_SeqId_46_ into 99b453beb9cd8e4a604b7044c921115b/family1 2023-11-28 03:54:45,920 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HStore(855): Successfully loaded hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__ughu86451cu6b4p8ipn8uj3th5jmo6u93p19j7mfm5ive9t58r24ouh8ic0h095u/family1/b2e227eea609419b972aab74f6937cb0 into 99b453beb9cd8e4a604b7044c921115b/family1 (new location: hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/99b453beb9cd8e4a604b7044c921115b/family1/2c79e042b8c642dd81727fb552b3c45c_SeqId_46_) 2023-11-28 03:54:45,921 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__ughu86451cu6b4p8ipn8uj3th5jmo6u93p19j7mfm5ive9t58r24ouh8ic0h095u/family1/b2e227eea609419b972aab74f6937cb0 2023-11-28 03:54:45,923 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-11-28 03:54:45,923 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-28 03:54:45,924 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201511303c5045422efa4979805652ad718fa56f to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201511303c5045422efa4979805652ad718fa56f 2023-11-28 03:54:45,925 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201512010e622c27fb3a44b88523a6abe10940e0 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201512010e622c27fb3a44b88523a6abe10940e0 2023-11-28 03:54:45,926 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2015120583ef133e9ae9427da9b26cc0443d5fe2 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2015120583ef133e9ae9427da9b26cc0443d5fe2 2023-11-28 03:54:45,926 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 3 expired mob files are deleted 2023-11-28 03:54:45,926 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151130' 2023-11-28 03:54:45,927 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-11-28 03:54:45,927 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] compactions.PartitionedMobCompactor(354): After compaction, there are 5 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-11-28 03:54:45,927 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-11-28 03:54:45,928 DEBUG [PEWorker-5] locking.LockProcedure(242): UNLOCKED pid=58, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-11-28 03:54:45,929 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=58, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE in 1.0420 sec 2023-11-28 03:54:46,003 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.HMaster$12(2454): Client=jenkins//172.31.14.131 modify table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily from 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} to 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} 2023-11-28 03:54:46,004 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] procedure2.ProcedureExecutor(1028): Stored pid=59, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:46,269 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/99b453beb9cd8e4a604b7044c921115b/.tmp/family1/592bf82fb4f3402fbfd472418496aafd as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/99b453beb9cd8e4a604b7044c921115b/family1/592bf82fb4f3402fbfd472418496aafd 2023-11-28 03:54:46,273 INFO [RS:0;jenkins-hbase4:41203-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in 99b453beb9cd8e4a604b7044c921115b/family1 of 99b453beb9cd8e4a604b7044c921115b into 592bf82fb4f3402fbfd472418496aafd(size=6.8 K), total size for store is 388.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-11-28 03:54:46,274 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 99b453beb9cd8e4a604b7044c921115b: 2023-11-28 03:54:46,274 INFO [RS:0;jenkins-hbase4:41203-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701143673936.99b453beb9cd8e4a604b7044c921115b., storeName=family1, fileCount=10, fileSize=50.3 K (5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.6 K), priority=2, time=1701143685410; duration=0sec 2023-11-28 03:54:46,274 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-28 03:54:46,291 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/99b453beb9cd8e4a604b7044c921115b/family1/39e59533461c4a9abaa084c693cdb9f6, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/99b453beb9cd8e4a604b7044c921115b/family1/c222c6b504d843178d7372b76072fd12, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/99b453beb9cd8e4a604b7044c921115b/family1/29b42335b68149bf8b264586e11860e6, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/99b453beb9cd8e4a604b7044c921115b/family1/a004d47b16bf4569885888eab5247319, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/99b453beb9cd8e4a604b7044c921115b/family1/0bd66c99f1df4a31a3e23a20dbafb992, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/99b453beb9cd8e4a604b7044c921115b/family1/47986f152f544c6a9851924a991b88cc, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/99b453beb9cd8e4a604b7044c921115b/family1/67614a2883774e5d8d6500a2da56638a, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/99b453beb9cd8e4a604b7044c921115b/family1/aa9a6fe34de641d08f12a1d8bb92c3d1, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/99b453beb9cd8e4a604b7044c921115b/family1/02380d42bbf14106ae64f93c2bb53754, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/99b453beb9cd8e4a604b7044c921115b/family1/c2881aa614184c9bb062c7b74f497cc5_SeqId_44_] to archive 2023-11-28 03:54:46,292 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-28 03:54:46,294 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/99b453beb9cd8e4a604b7044c921115b/family1/39e59533461c4a9abaa084c693cdb9f6 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/99b453beb9cd8e4a604b7044c921115b/family1/39e59533461c4a9abaa084c693cdb9f6 2023-11-28 03:54:46,295 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/99b453beb9cd8e4a604b7044c921115b/family1/c222c6b504d843178d7372b76072fd12 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/99b453beb9cd8e4a604b7044c921115b/family1/c222c6b504d843178d7372b76072fd12 2023-11-28 03:54:46,296 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/99b453beb9cd8e4a604b7044c921115b/family1/29b42335b68149bf8b264586e11860e6 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/99b453beb9cd8e4a604b7044c921115b/family1/29b42335b68149bf8b264586e11860e6 2023-11-28 03:54:46,297 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/99b453beb9cd8e4a604b7044c921115b/family1/a004d47b16bf4569885888eab5247319 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/99b453beb9cd8e4a604b7044c921115b/family1/a004d47b16bf4569885888eab5247319 2023-11-28 03:54:46,298 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/99b453beb9cd8e4a604b7044c921115b/family1/0bd66c99f1df4a31a3e23a20dbafb992 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/99b453beb9cd8e4a604b7044c921115b/family1/0bd66c99f1df4a31a3e23a20dbafb992 2023-11-28 03:54:46,299 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/99b453beb9cd8e4a604b7044c921115b/family1/47986f152f544c6a9851924a991b88cc to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/99b453beb9cd8e4a604b7044c921115b/family1/47986f152f544c6a9851924a991b88cc 2023-11-28 03:54:46,300 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/99b453beb9cd8e4a604b7044c921115b/family1/67614a2883774e5d8d6500a2da56638a to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/99b453beb9cd8e4a604b7044c921115b/family1/67614a2883774e5d8d6500a2da56638a 2023-11-28 03:54:46,302 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/99b453beb9cd8e4a604b7044c921115b/family1/aa9a6fe34de641d08f12a1d8bb92c3d1 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/99b453beb9cd8e4a604b7044c921115b/family1/aa9a6fe34de641d08f12a1d8bb92c3d1 2023-11-28 03:54:46,303 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/99b453beb9cd8e4a604b7044c921115b/family1/02380d42bbf14106ae64f93c2bb53754 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/99b453beb9cd8e4a604b7044c921115b/family1/02380d42bbf14106ae64f93c2bb53754 2023-11-28 03:54:46,305 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/99b453beb9cd8e4a604b7044c921115b/family1/c2881aa614184c9bb062c7b74f497cc5_SeqId_44_ to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/99b453beb9cd8e4a604b7044c921115b/family1/c2881aa614184c9bb062c7b74f497cc5_SeqId_44_ 2023-11-28 03:54:46,417 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000002 2023-11-28 03:54:46,418 DEBUG [PEWorker-3] util.FSTableDescriptors(519): Deleted hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000001 2023-11-28 03:54:46,418 INFO [PEWorker-3] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000002 2023-11-28 03:54:46,420 INFO [PEWorker-3] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=60, ppid=59, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily}] 2023-11-28 03:54:46,422 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=61, ppid=60, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=99b453beb9cd8e4a604b7044c921115b, REOPEN/MOVE}] 2023-11-28 03:54:46,422 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=61, ppid=60, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=99b453beb9cd8e4a604b7044c921115b, REOPEN/MOVE 2023-11-28 03:54:46,423 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=61 updating hbase:meta row=99b453beb9cd8e4a604b7044c921115b, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:46,423 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701143673936.99b453beb9cd8e4a604b7044c921115b.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1701143686423"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1701143686423"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1701143686423"}]},"ts":"1701143686423"} 2023-11-28 03:54:46,425 INFO [PEWorker-2] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=62, ppid=61, state=RUNNABLE; CloseRegionProcedure 99b453beb9cd8e4a604b7044c921115b, server=jenkins-hbase4.apache.org,41203,1701143600653}] 2023-11-28 03:54:46,576 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:46,576 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 99b453beb9cd8e4a604b7044c921115b 2023-11-28 03:54:46,577 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 99b453beb9cd8e4a604b7044c921115b, disabling compactions & flushes 2023-11-28 03:54:46,577 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701143673936.99b453beb9cd8e4a604b7044c921115b. 2023-11-28 03:54:46,577 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701143673936.99b453beb9cd8e4a604b7044c921115b. 2023-11-28 03:54:46,577 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701143673936.99b453beb9cd8e4a604b7044c921115b. after waiting 0 ms 2023-11-28 03:54:46,577 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701143673936.99b453beb9cd8e4a604b7044c921115b. 2023-11-28 03:54:46,583 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/99b453beb9cd8e4a604b7044c921115b/recovered.edits/49.seqid, newMaxSeqId=49, maxSeqId=1 2023-11-28 03:54:46,584 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701143673936.99b453beb9cd8e4a604b7044c921115b. 2023-11-28 03:54:46,584 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 99b453beb9cd8e4a604b7044c921115b: 2023-11-28 03:54:46,584 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3512): Not adding moved region record: 99b453beb9cd8e4a604b7044c921115b to self. 2023-11-28 03:54:46,585 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 99b453beb9cd8e4a604b7044c921115b 2023-11-28 03:54:46,586 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=61 updating hbase:meta row=99b453beb9cd8e4a604b7044c921115b, regionState=CLOSED 2023-11-28 03:54:46,586 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701143673936.99b453beb9cd8e4a604b7044c921115b.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1701143686586"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1701143686586"}]},"ts":"1701143686586"} 2023-11-28 03:54:46,588 INFO [PEWorker-5] procedure2.ProcedureExecutor(1825): Finished subprocedure pid=62, resume processing ppid=61 2023-11-28 03:54:46,588 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=62, ppid=61, state=SUCCESS; CloseRegionProcedure 99b453beb9cd8e4a604b7044c921115b, server=jenkins-hbase4.apache.org,41203,1701143600653 in 162 msec 2023-11-28 03:54:46,589 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=61, ppid=60, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=99b453beb9cd8e4a604b7044c921115b, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,41203,1701143600653; forceNewPlan=false, retain=true 2023-11-28 03:54:46,739 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=61 updating hbase:meta row=99b453beb9cd8e4a604b7044c921115b, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:46,739 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701143673936.99b453beb9cd8e4a604b7044c921115b.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1701143686739"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1701143686739"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1701143686739"}]},"ts":"1701143686739"} 2023-11-28 03:54:46,741 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=63, ppid=61, state=RUNNABLE; OpenRegionProcedure 99b453beb9cd8e4a604b7044c921115b, server=jenkins-hbase4.apache.org,41203,1701143600653}] 2023-11-28 03:54:46,892 DEBUG [RSProcedureDispatcher-pool-1] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:46,895 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701143673936.99b453beb9cd8e4a604b7044c921115b. 2023-11-28 03:54:46,895 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 99b453beb9cd8e4a604b7044c921115b, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701143673936.99b453beb9cd8e4a604b7044c921115b.', STARTKEY => '', ENDKEY => ''} 2023-11-28 03:54:46,896 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 99b453beb9cd8e4a604b7044c921115b 2023-11-28 03:54:46,896 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701143673936.99b453beb9cd8e4a604b7044c921115b.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-28 03:54:46,896 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 99b453beb9cd8e4a604b7044c921115b 2023-11-28 03:54:46,896 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 99b453beb9cd8e4a604b7044c921115b 2023-11-28 03:54:46,897 INFO [StoreOpener-99b453beb9cd8e4a604b7044c921115b-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 99b453beb9cd8e4a604b7044c921115b 2023-11-28 03:54:46,898 DEBUG [StoreOpener-99b453beb9cd8e4a604b7044c921115b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/99b453beb9cd8e4a604b7044c921115b/family1 2023-11-28 03:54:46,898 DEBUG [StoreOpener-99b453beb9cd8e4a604b7044c921115b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/99b453beb9cd8e4a604b7044c921115b/family1 2023-11-28 03:54:46,898 INFO [StoreOpener-99b453beb9cd8e4a604b7044c921115b-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 99b453beb9cd8e4a604b7044c921115b columnFamilyName family1 2023-11-28 03:54:46,904 DEBUG [StoreOpener-99b453beb9cd8e4a604b7044c921115b-1] regionserver.HStore(539): loaded hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/99b453beb9cd8e4a604b7044c921115b/family1/2c79e042b8c642dd81727fb552b3c45c_SeqId_46_ 2023-11-28 03:54:46,908 DEBUG [StoreOpener-99b453beb9cd8e4a604b7044c921115b-1] regionserver.HStore(539): loaded hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/99b453beb9cd8e4a604b7044c921115b/family1/592bf82fb4f3402fbfd472418496aafd 2023-11-28 03:54:46,911 DEBUG [StoreOpener-99b453beb9cd8e4a604b7044c921115b-1] regionserver.HStore(539): loaded hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/99b453beb9cd8e4a604b7044c921115b/family1/87ce5a41564c45e788ba9ea6bc5ee862 2023-11-28 03:54:46,911 INFO [StoreOpener-99b453beb9cd8e4a604b7044c921115b-1] regionserver.HStore(310): Store=99b453beb9cd8e4a604b7044c921115b/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-28 03:54:46,912 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/99b453beb9cd8e4a604b7044c921115b 2023-11-28 03:54:46,913 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/99b453beb9cd8e4a604b7044c921115b 2023-11-28 03:54:46,915 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 99b453beb9cd8e4a604b7044c921115b 2023-11-28 03:54:46,916 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 99b453beb9cd8e4a604b7044c921115b; next sequenceid=50; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60750846, jitterRate=-0.09474185109138489}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-11-28 03:54:46,916 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 99b453beb9cd8e4a604b7044c921115b: 2023-11-28 03:54:46,917 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2339): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701143673936.99b453beb9cd8e4a604b7044c921115b., pid=63, masterSystemTime=1701143686892 2023-11-28 03:54:46,917 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-28 03:54:46,917 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-11-28 03:54:46,919 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 397422 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-11-28 03:54:46,919 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] regionserver.HStore(1912): 99b453beb9cd8e4a604b7044c921115b/family1 is initiating minor compaction (all files) 2023-11-28 03:54:46,919 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2366): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701143673936.99b453beb9cd8e4a604b7044c921115b. 2023-11-28 03:54:46,919 INFO [RS:0;jenkins-hbase4:41203-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 99b453beb9cd8e4a604b7044c921115b/family1 in testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701143673936.99b453beb9cd8e4a604b7044c921115b. 2023-11-28 03:54:46,919 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701143673936.99b453beb9cd8e4a604b7044c921115b. 2023-11-28 03:54:46,919 INFO [RS:0;jenkins-hbase4:41203-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/99b453beb9cd8e4a604b7044c921115b/family1/87ce5a41564c45e788ba9ea6bc5ee862, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/99b453beb9cd8e4a604b7044c921115b/family1/592bf82fb4f3402fbfd472418496aafd, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/99b453beb9cd8e4a604b7044c921115b/family1/2c79e042b8c642dd81727fb552b3c45c_SeqId_46_] into tmpdir=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/99b453beb9cd8e4a604b7044c921115b/.tmp, totalSize=388.1 K 2023-11-28 03:54:46,919 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=61 updating hbase:meta row=99b453beb9cd8e4a604b7044c921115b, regionState=OPEN, openSeqNum=50, regionLocation=jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:46,920 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701143673936.99b453beb9cd8e4a604b7044c921115b.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1701143686919"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1701143686919"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1701143686919"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1701143686919"}]},"ts":"1701143686919"} 2023-11-28 03:54:46,920 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] compactions.Compactor(207): Compacting 87ce5a41564c45e788ba9ea6bc5ee862, keycount=1000, bloomtype=ROW, size=190.5 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878824053 2023-11-28 03:54:46,920 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] compactions.Compactor(207): Compacting 592bf82fb4f3402fbfd472418496aafd, keycount=9, bloomtype=ROW, size=6.8 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621224053 2023-11-28 03:54:46,921 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] compactions.Compactor(207): Compacting 2c79e042b8c642dd81727fb552b3c45c_SeqId_46_, keycount=1002, bloomtype=ROW, size=190.9 K, encoding=NONE, compression=NONE, seqNum=47, earliestPutTs=1448878824053 2023-11-28 03:54:46,922 INFO [PEWorker-4] procedure2.ProcedureExecutor(1825): Finished subprocedure pid=63, resume processing ppid=61 2023-11-28 03:54:46,922 INFO [PEWorker-4] procedure2.ProcedureExecutor(1409): Finished pid=63, ppid=61, state=SUCCESS; OpenRegionProcedure 99b453beb9cd8e4a604b7044c921115b, server=jenkins-hbase4.apache.org,41203,1701143600653 in 180 msec 2023-11-28 03:54:46,923 INFO [PEWorker-5] procedure2.ProcedureExecutor(1825): Finished subprocedure pid=61, resume processing ppid=60 2023-11-28 03:54:46,924 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=61, ppid=60, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=99b453beb9cd8e4a604b7044c921115b, REOPEN/MOVE in 500 msec 2023-11-28 03:54:46,925 INFO [PEWorker-3] procedure2.ProcedureExecutor(1825): Finished subprocedure pid=60, resume processing ppid=59 2023-11-28 03:54:46,926 INFO [PEWorker-3] procedure2.ProcedureExecutor(1409): Finished pid=60, ppid=59, state=SUCCESS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily in 504 msec 2023-11-28 03:54:46,927 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=59, state=SUCCESS; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily in 923 msec 2023-11-28 03:54:46,928 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(1230): Checking to see if procedure is done pid=59 2023-11-28 03:54:46,929 INFO [Listener at localhost/42853] compactions.TestMobCompactor(278): alter status finished 2023-11-28 03:54:46,930 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=38593] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:46,931 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] procedure2.ProcedureExecutor(1028): Stored pid=64, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-11-28 03:54:46,932 DEBUG [PEWorker-2] locking.LockProcedure(309): LOCKED pid=64, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-11-28 03:54:46,933 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-11-28 03:54:46,933 INFO [PEWorker-2] procedure2.TimeoutExecutorThread(81): ADDED pid=64, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1701144286933 2023-11-28 03:54:46,933 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-11-28 03:54:46,934 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 5 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column 'family1' 2023-11-28 03:54:46,934 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-11-28 03:54:46,935 INFO [RS:0;jenkins-hbase4:41203-shortCompactions-0] throttle.PressureAwareThroughputController(145): 99b453beb9cd8e4a604b7044c921115b#family1#compaction#73 average throughput is 22.45 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-11-28 03:54:46,935 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] zookeeper.ReadOnlyZKClient(139): Connect 0x3f139172 to 127.0.0.1:55641 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-11-28 03:54:46,942 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] ipc.AbstractRpcClient(189): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@93496bd, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-11-28 03:54:46,942 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-11-28 03:54:46,943 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-11-28 03:54:46,943 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-11-28 03:54:46,943 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-11-28 03:54:46,943 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-11-28 03:54:47,766 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511285b4dd6b97df2432a87111cc18ebe531d to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201511285b4dd6b97df2432a87111cc18ebe531d 2023-11-28 03:54:47,766 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-11-28 03:54:47,779 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(122): Using SIMPLE authentication for service=ClientService, sasl=false 2023-11-28 03:54:47,780 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:42538, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-11-28 03:54:47,786 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(122): Using SIMPLE authentication for service=MasterService, sasl=false 2023-11-28 03:54:47,787 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:41974, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-11-28 03:54:47,795 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151101/family1/43612f35e9ae48498565d64db029ff1b first=Optional[r05] last=Optional[r8] 2023-11-28 03:54:47,795 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701143673936.99b453beb9cd8e4a604b7044c921115b., hostname=jenkins-hbase4.apache.org,41203,1701143600653, seqNum=50 for row with hfile group [{family1,hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151101/family1/43612f35e9ae48498565d64db029ff1b}] 2023-11-28 03:54:47,799 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HStore(763): Validating hfile at hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151101/family1/43612f35e9ae48498565d64db029ff1b for inclusion in 99b453beb9cd8e4a604b7044c921115b/family1 2023-11-28 03:54:47,801 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HStore(775): HFile bounds: first=r05 last=r8 2023-11-28 03:54:47,801 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HStore(777): Region bounds: first= last= 2023-11-28 03:54:47,801 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HRegion(2446): Flush status journal for 99b453beb9cd8e4a604b7044c921115b: 2023-11-28 03:54:47,802 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151101/family1/43612f35e9ae48498565d64db029ff1b to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__228t3iir4bs46fdggon0kqqvk00ltscje1nhtikb9q6hl1fcel4rnb1jdit3s9n5/family1/43612f35e9ae48498565d64db029ff1b 2023-11-28 03:54:47,804 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__228t3iir4bs46fdggon0kqqvk00ltscje1nhtikb9q6hl1fcel4rnb1jdit3s9n5/family1/43612f35e9ae48498565d64db029ff1b as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/99b453beb9cd8e4a604b7044c921115b/family1/50d8a6722ef848a985c689f82ae3a0c6_SeqId_52_ 2023-11-28 03:54:47,805 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HStore(849): Loaded HFile hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__228t3iir4bs46fdggon0kqqvk00ltscje1nhtikb9q6hl1fcel4rnb1jdit3s9n5/family1/43612f35e9ae48498565d64db029ff1b into 99b453beb9cd8e4a604b7044c921115b/family1 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/99b453beb9cd8e4a604b7044c921115b/family1/50d8a6722ef848a985c689f82ae3a0c6_SeqId_52_ - updating store file list. 2023-11-28 03:54:47,808 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HStore(882): Loaded HFile hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/99b453beb9cd8e4a604b7044c921115b/family1/50d8a6722ef848a985c689f82ae3a0c6_SeqId_52_ into 99b453beb9cd8e4a604b7044c921115b/family1 2023-11-28 03:54:47,808 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HStore(855): Successfully loaded hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__228t3iir4bs46fdggon0kqqvk00ltscje1nhtikb9q6hl1fcel4rnb1jdit3s9n5/family1/43612f35e9ae48498565d64db029ff1b into 99b453beb9cd8e4a604b7044c921115b/family1 (new location: hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/99b453beb9cd8e4a604b7044c921115b/family1/50d8a6722ef848a985c689f82ae3a0c6_SeqId_52_) 2023-11-28 03:54:47,808 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__228t3iir4bs46fdggon0kqqvk00ltscje1nhtikb9q6hl1fcel4rnb1jdit3s9n5/family1/43612f35e9ae48498565d64db029ff1b 2023-11-28 03:54:47,810 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-11-28 03:54:47,811 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-28 03:54:47,812 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201511209be19f54a2bd451db9c2f3b2fffa42b0 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201511209be19f54a2bd451db9c2f3b2fffa42b0 2023-11-28 03:54:47,813 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201511283c07c5a0046b4bc683845aad7298321c to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201511283c07c5a0046b4bc683845aad7298321c 2023-11-28 03:54:47,813 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2 expired mob files are deleted 2023-11-28 03:54:47,813 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151101' 2023-11-28 03:54:47,813 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-11-28 03:54:47,813 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-11-28 03:54:47,813 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-11-28 03:54:47,813 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-11-28 03:54:47,813 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-11-28 03:54:47,814 DEBUG [PEWorker-4] locking.LockProcedure(242): UNLOCKED pid=64, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-11-28 03:54:47,815 INFO [PEWorker-4] procedure2.ProcedureExecutor(1409): Finished pid=64, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE in 883 msec 2023-11-28 03:54:47,869 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.HMaster$12(2454): Client=jenkins//172.31.14.131 modify table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily from 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} to 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'DAILY'} 2023-11-28 03:54:47,870 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] procedure2.ProcedureExecutor(1028): Stored pid=65, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:48,153 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/99b453beb9cd8e4a604b7044c921115b/.tmp/family1/d6a49ae4bdb14869aca92e107d3fd2eb as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/99b453beb9cd8e4a604b7044c921115b/family1/d6a49ae4bdb14869aca92e107d3fd2eb 2023-11-28 03:54:48,158 INFO [RS:0;jenkins-hbase4:41203-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 99b453beb9cd8e4a604b7044c921115b/family1 of 99b453beb9cd8e4a604b7044c921115b into d6a49ae4bdb14869aca92e107d3fd2eb(size=192.0 K), total size for store is 197.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-11-28 03:54:48,158 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 99b453beb9cd8e4a604b7044c921115b: 2023-11-28 03:54:48,158 INFO [RS:0;jenkins-hbase4:41203-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701143673936.99b453beb9cd8e4a604b7044c921115b., storeName=99b453beb9cd8e4a604b7044c921115b/family1, priority=13, startTime=1701143686917; duration=1sec 2023-11-28 03:54:48,158 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-28 03:54:48,178 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/99b453beb9cd8e4a604b7044c921115b/family1/87ce5a41564c45e788ba9ea6bc5ee862, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/99b453beb9cd8e4a604b7044c921115b/family1/592bf82fb4f3402fbfd472418496aafd, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/99b453beb9cd8e4a604b7044c921115b/family1/2c79e042b8c642dd81727fb552b3c45c_SeqId_46_] to archive 2023-11-28 03:54:48,178 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-28 03:54:48,180 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/99b453beb9cd8e4a604b7044c921115b/family1/87ce5a41564c45e788ba9ea6bc5ee862 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/99b453beb9cd8e4a604b7044c921115b/family1/87ce5a41564c45e788ba9ea6bc5ee862 2023-11-28 03:54:48,181 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/99b453beb9cd8e4a604b7044c921115b/family1/592bf82fb4f3402fbfd472418496aafd to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/99b453beb9cd8e4a604b7044c921115b/family1/592bf82fb4f3402fbfd472418496aafd 2023-11-28 03:54:48,183 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/99b453beb9cd8e4a604b7044c921115b/family1/2c79e042b8c642dd81727fb552b3c45c_SeqId_46_ to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/99b453beb9cd8e4a604b7044c921115b/family1/2c79e042b8c642dd81727fb552b3c45c_SeqId_46_ 2023-11-28 03:54:48,278 DEBUG [PEWorker-5] util.FSTableDescriptors(570): Wrote into hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000003 2023-11-28 03:54:48,279 DEBUG [PEWorker-5] util.FSTableDescriptors(519): Deleted hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000002 2023-11-28 03:54:48,279 INFO [PEWorker-5] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000003 2023-11-28 03:54:48,281 INFO [PEWorker-5] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=66, ppid=65, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily}] 2023-11-28 03:54:48,283 INFO [PEWorker-3] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=67, ppid=66, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=99b453beb9cd8e4a604b7044c921115b, REOPEN/MOVE}] 2023-11-28 03:54:48,284 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=67, ppid=66, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=99b453beb9cd8e4a604b7044c921115b, REOPEN/MOVE 2023-11-28 03:54:48,284 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=67 updating hbase:meta row=99b453beb9cd8e4a604b7044c921115b, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:48,284 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701143673936.99b453beb9cd8e4a604b7044c921115b.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1701143688284"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1701143688284"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1701143688284"}]},"ts":"1701143688284"} 2023-11-28 03:54:48,286 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=68, ppid=67, state=RUNNABLE; CloseRegionProcedure 99b453beb9cd8e4a604b7044c921115b, server=jenkins-hbase4.apache.org,41203,1701143600653}] 2023-11-28 03:54:48,437 DEBUG [RSProcedureDispatcher-pool-2] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:48,438 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 99b453beb9cd8e4a604b7044c921115b 2023-11-28 03:54:48,438 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 99b453beb9cd8e4a604b7044c921115b, disabling compactions & flushes 2023-11-28 03:54:48,438 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701143673936.99b453beb9cd8e4a604b7044c921115b. 2023-11-28 03:54:48,438 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701143673936.99b453beb9cd8e4a604b7044c921115b. 2023-11-28 03:54:48,438 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701143673936.99b453beb9cd8e4a604b7044c921115b. after waiting 0 ms 2023-11-28 03:54:48,438 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701143673936.99b453beb9cd8e4a604b7044c921115b. 2023-11-28 03:54:48,446 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/99b453beb9cd8e4a604b7044c921115b/recovered.edits/55.seqid, newMaxSeqId=55, maxSeqId=49 2023-11-28 03:54:48,446 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701143673936.99b453beb9cd8e4a604b7044c921115b. 2023-11-28 03:54:48,446 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 99b453beb9cd8e4a604b7044c921115b: 2023-11-28 03:54:48,446 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3512): Not adding moved region record: 99b453beb9cd8e4a604b7044c921115b to self. 2023-11-28 03:54:48,448 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 99b453beb9cd8e4a604b7044c921115b 2023-11-28 03:54:48,448 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=67 updating hbase:meta row=99b453beb9cd8e4a604b7044c921115b, regionState=CLOSED 2023-11-28 03:54:48,448 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701143673936.99b453beb9cd8e4a604b7044c921115b.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1701143688448"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1701143688448"}]},"ts":"1701143688448"} 2023-11-28 03:54:48,450 INFO [PEWorker-4] procedure2.ProcedureExecutor(1825): Finished subprocedure pid=68, resume processing ppid=67 2023-11-28 03:54:48,450 INFO [PEWorker-4] procedure2.ProcedureExecutor(1409): Finished pid=68, ppid=67, state=SUCCESS; CloseRegionProcedure 99b453beb9cd8e4a604b7044c921115b, server=jenkins-hbase4.apache.org,41203,1701143600653 in 163 msec 2023-11-28 03:54:48,451 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=67, ppid=66, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=99b453beb9cd8e4a604b7044c921115b, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,41203,1701143600653; forceNewPlan=false, retain=true 2023-11-28 03:54:48,601 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=67 updating hbase:meta row=99b453beb9cd8e4a604b7044c921115b, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:48,601 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701143673936.99b453beb9cd8e4a604b7044c921115b.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1701143688601"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1701143688601"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1701143688601"}]},"ts":"1701143688601"} 2023-11-28 03:54:48,603 INFO [PEWorker-3] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=69, ppid=67, state=RUNNABLE; OpenRegionProcedure 99b453beb9cd8e4a604b7044c921115b, server=jenkins-hbase4.apache.org,41203,1701143600653}] 2023-11-28 03:54:48,755 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:48,758 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701143673936.99b453beb9cd8e4a604b7044c921115b. 2023-11-28 03:54:48,758 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 99b453beb9cd8e4a604b7044c921115b, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701143673936.99b453beb9cd8e4a604b7044c921115b.', STARTKEY => '', ENDKEY => ''} 2023-11-28 03:54:48,758 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 99b453beb9cd8e4a604b7044c921115b 2023-11-28 03:54:48,758 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701143673936.99b453beb9cd8e4a604b7044c921115b.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-28 03:54:48,758 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 99b453beb9cd8e4a604b7044c921115b 2023-11-28 03:54:48,758 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 99b453beb9cd8e4a604b7044c921115b 2023-11-28 03:54:48,760 INFO [StoreOpener-99b453beb9cd8e4a604b7044c921115b-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 99b453beb9cd8e4a604b7044c921115b 2023-11-28 03:54:48,760 DEBUG [StoreOpener-99b453beb9cd8e4a604b7044c921115b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/99b453beb9cd8e4a604b7044c921115b/family1 2023-11-28 03:54:48,761 DEBUG [StoreOpener-99b453beb9cd8e4a604b7044c921115b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/99b453beb9cd8e4a604b7044c921115b/family1 2023-11-28 03:54:48,761 INFO [StoreOpener-99b453beb9cd8e4a604b7044c921115b-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 99b453beb9cd8e4a604b7044c921115b columnFamilyName family1 2023-11-28 03:54:48,766 DEBUG [StoreOpener-99b453beb9cd8e4a604b7044c921115b-1] regionserver.HStore(539): loaded hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/99b453beb9cd8e4a604b7044c921115b/family1/50d8a6722ef848a985c689f82ae3a0c6_SeqId_52_ 2023-11-28 03:54:48,769 DEBUG [StoreOpener-99b453beb9cd8e4a604b7044c921115b-1] regionserver.HStore(539): loaded hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/99b453beb9cd8e4a604b7044c921115b/family1/d6a49ae4bdb14869aca92e107d3fd2eb 2023-11-28 03:54:48,769 INFO [StoreOpener-99b453beb9cd8e4a604b7044c921115b-1] regionserver.HStore(310): Store=99b453beb9cd8e4a604b7044c921115b/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-28 03:54:48,770 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/99b453beb9cd8e4a604b7044c921115b 2023-11-28 03:54:48,771 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/99b453beb9cd8e4a604b7044c921115b 2023-11-28 03:54:48,773 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 99b453beb9cd8e4a604b7044c921115b 2023-11-28 03:54:48,774 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 99b453beb9cd8e4a604b7044c921115b; next sequenceid=56; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61608151, jitterRate=-0.08196701109409332}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-11-28 03:54:48,774 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 99b453beb9cd8e4a604b7044c921115b: 2023-11-28 03:54:48,775 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2339): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701143673936.99b453beb9cd8e4a604b7044c921115b., pid=69, masterSystemTime=1701143688755 2023-11-28 03:54:48,776 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2366): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701143673936.99b453beb9cd8e4a604b7044c921115b. 2023-11-28 03:54:48,776 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701143673936.99b453beb9cd8e4a604b7044c921115b. 2023-11-28 03:54:48,776 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=67 updating hbase:meta row=99b453beb9cd8e4a604b7044c921115b, regionState=OPEN, openSeqNum=56, regionLocation=jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:48,776 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701143673936.99b453beb9cd8e4a604b7044c921115b.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1701143688776"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1701143688776"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1701143688776"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1701143688776"}]},"ts":"1701143688776"} 2023-11-28 03:54:48,779 INFO [PEWorker-2] procedure2.ProcedureExecutor(1825): Finished subprocedure pid=69, resume processing ppid=67 2023-11-28 03:54:48,779 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=69, ppid=67, state=SUCCESS; OpenRegionProcedure 99b453beb9cd8e4a604b7044c921115b, server=jenkins-hbase4.apache.org,41203,1701143600653 in 174 msec 2023-11-28 03:54:48,780 INFO [PEWorker-4] procedure2.ProcedureExecutor(1825): Finished subprocedure pid=67, resume processing ppid=66 2023-11-28 03:54:48,780 INFO [PEWorker-4] procedure2.ProcedureExecutor(1409): Finished pid=67, ppid=66, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=99b453beb9cd8e4a604b7044c921115b, REOPEN/MOVE in 496 msec 2023-11-28 03:54:48,782 INFO [PEWorker-5] procedure2.ProcedureExecutor(1825): Finished subprocedure pid=66, resume processing ppid=65 2023-11-28 03:54:48,782 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=66, ppid=65, state=SUCCESS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily in 500 msec 2023-11-28 03:54:48,783 INFO [PEWorker-3] procedure2.ProcedureExecutor(1409): Finished pid=65, state=SUCCESS; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily in 913 msec 2023-11-28 03:54:48,784 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(1230): Checking to see if procedure is done pid=65 2023-11-28 03:54:48,785 INFO [Listener at localhost/42853] compactions.TestMobCompactor(278): alter status finished 2023-11-28 03:54:48,786 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=38593] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'DAILY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-28 03:54:48,787 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] procedure2.ProcedureExecutor(1028): Stored pid=70, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-11-28 03:54:48,787 DEBUG [PEWorker-1] locking.LockProcedure(309): LOCKED pid=70, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-11-28 03:54:48,788 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-11-28 03:54:48,788 INFO [PEWorker-1] procedure2.TimeoutExecutorThread(81): ADDED pid=70, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1701144288788 2023-11-28 03:54:48,789 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-11-28 03:54:48,789 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 4 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column 'family1' 2023-11-28 03:54:48,789 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-11-28 03:54:48,790 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] zookeeper.ReadOnlyZKClient(139): Connect 0x3c94566c to 127.0.0.1:55641 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-11-28 03:54:48,795 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] ipc.AbstractRpcClient(189): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@28b0b016, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-11-28 03:54:48,795 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151128 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-11-28 03:54:48,795 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151128 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-11-28 03:54:48,795 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151205 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-11-28 03:54:48,795 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151205 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-11-28 03:54:48,795 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160103 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-11-28 03:54:48,795 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160103 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-11-28 03:54:48,795 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-11-28 03:54:48,795 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-11-28 03:54:48,796 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-11-28 03:54:48,796 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-11-28 03:54:48,796 DEBUG [PEWorker-2] locking.LockProcedure(242): UNLOCKED pid=70, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-11-28 03:54:48,797 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=70, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE in 10 msec 2023-11-28 03:54:48,852 INFO [Listener at localhost/42853] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily Thread=245 (was 248), OpenFileDescriptor=645 (was 652), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=170 (was 169) - SystemLoadAverage LEAK? -, ProcessCount=170 (was 172), AvailableMemoryMB=6913 (was 6963) 2023-11-28 03:54:48,863 INFO [Listener at localhost/42853] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testGetAfterCompaction Thread=245, OpenFileDescriptor=645, MaxFileDescriptor=60000, SystemLoadAverage=170, ProcessCount=170, AvailableMemoryMB=6912 2023-11-28 03:54:48,866 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testGetAfterCompaction', {TABLE_ATTRIBUTES => {coprocessor$1 => '|org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro|1073741823|'}}, {NAME => 'f1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} 2023-11-28 03:54:48,867 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] procedure2.ProcedureExecutor(1028): Stored pid=71, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testGetAfterCompaction 2023-11-28 03:54:48,868 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_PRE_OPERATION 2023-11-28 03:54:48,868 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(700): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testGetAfterCompaction" procId is: 71 2023-11-28 03:54:48,868 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-11-28 03:54:48,869 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(1230): Checking to see if procedure is done pid=71 2023-11-28 03:54:48,870 DEBUG [HFileArchiver-11] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/.tmp/data/default/testGetAfterCompaction/493f75b50f64227c277f1462ad94de11 2023-11-28 03:54:48,870 DEBUG [HFileArchiver-11] backup.HFileArchiver(153): Directory hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/.tmp/data/default/testGetAfterCompaction/493f75b50f64227c277f1462ad94de11 empty. 2023-11-28 03:54:48,871 DEBUG [HFileArchiver-11] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/.tmp/data/default/testGetAfterCompaction/493f75b50f64227c277f1462ad94de11 2023-11-28 03:54:48,871 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(328): Archived testGetAfterCompaction regions 2023-11-28 03:54:48,970 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(1230): Checking to see if procedure is done pid=71 2023-11-28 03:54:49,171 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(1230): Checking to see if procedure is done pid=71 2023-11-28 03:54:49,278 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/.tmp/data/default/testGetAfterCompaction/.tabledesc/.tableinfo.0000000001 2023-11-28 03:54:49,280 INFO [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(7675): creating {ENCODED => 493f75b50f64227c277f1462ad94de11, NAME => 'testGetAfterCompaction,,1701143688866.493f75b50f64227c277f1462ad94de11.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testGetAfterCompaction', {TABLE_ATTRIBUTES => {coprocessor$1 => '|org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro|1073741823|'}}, {NAME => 'f1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/.tmp 2023-11-28 03:54:49,472 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(1230): Checking to see if procedure is done pid=71 2023-11-28 03:54:49,685 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(866): Instantiated testGetAfterCompaction,,1701143688866.493f75b50f64227c277f1462ad94de11.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-28 03:54:49,686 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1604): Closing 493f75b50f64227c277f1462ad94de11, disabling compactions & flushes 2023-11-28 03:54:49,686 INFO [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1626): Closing region testGetAfterCompaction,,1701143688866.493f75b50f64227c277f1462ad94de11. 2023-11-28 03:54:49,686 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testGetAfterCompaction,,1701143688866.493f75b50f64227c277f1462ad94de11. 2023-11-28 03:54:49,686 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1714): Acquired close lock on testGetAfterCompaction,,1701143688866.493f75b50f64227c277f1462ad94de11. after waiting 0 ms 2023-11-28 03:54:49,686 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1724): Updates disabled for region testGetAfterCompaction,,1701143688866.493f75b50f64227c277f1462ad94de11. 2023-11-28 03:54:49,686 INFO [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1838): Closed testGetAfterCompaction,,1701143688866.493f75b50f64227c277f1462ad94de11. 2023-11-28 03:54:49,686 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1558): Region close journal for 493f75b50f64227c277f1462ad94de11: 2023-11-28 03:54:49,688 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_ADD_TO_META 2023-11-28 03:54:49,689 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testGetAfterCompaction,,1701143688866.493f75b50f64227c277f1462ad94de11.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1701143689689"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1701143689689"}]},"ts":"1701143689689"} 2023-11-28 03:54:49,690 INFO [PEWorker-4] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-11-28 03:54:49,691 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-11-28 03:54:49,691 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testGetAfterCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1701143689691"}]},"ts":"1701143689691"} 2023-11-28 03:54:49,692 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testGetAfterCompaction, state=ENABLING in hbase:meta 2023-11-28 03:54:49,695 INFO [PEWorker-4] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=72, ppid=71, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testGetAfterCompaction, region=493f75b50f64227c277f1462ad94de11, ASSIGN}] 2023-11-28 03:54:49,696 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=72, ppid=71, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testGetAfterCompaction, region=493f75b50f64227c277f1462ad94de11, ASSIGN 2023-11-28 03:54:49,696 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=72, ppid=71, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testGetAfterCompaction, region=493f75b50f64227c277f1462ad94de11, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,41203,1701143600653; forceNewPlan=false, retain=false 2023-11-28 03:54:49,847 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=72 updating hbase:meta row=493f75b50f64227c277f1462ad94de11, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:49,848 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testGetAfterCompaction,,1701143688866.493f75b50f64227c277f1462ad94de11.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1701143689847"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1701143689847"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1701143689847"}]},"ts":"1701143689847"} 2023-11-28 03:54:49,849 INFO [PEWorker-3] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=73, ppid=72, state=RUNNABLE; OpenRegionProcedure 493f75b50f64227c277f1462ad94de11, server=jenkins-hbase4.apache.org,41203,1701143600653}] 2023-11-28 03:54:49,973 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(1230): Checking to see if procedure is done pid=71 2023-11-28 03:54:50,001 DEBUG [RSProcedureDispatcher-pool-1] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:50,004 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testGetAfterCompaction,,1701143688866.493f75b50f64227c277f1462ad94de11. 2023-11-28 03:54:50,004 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 493f75b50f64227c277f1462ad94de11, NAME => 'testGetAfterCompaction,,1701143688866.493f75b50f64227c277f1462ad94de11.', STARTKEY => '', ENDKEY => ''} 2023-11-28 03:54:50,004 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro with path null and priority 1073741823 2023-11-28 03:54:50,006 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro from HTD of testGetAfterCompaction successfully. 2023-11-28 03:54:50,006 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testGetAfterCompaction 493f75b50f64227c277f1462ad94de11 2023-11-28 03:54:50,006 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testGetAfterCompaction,,1701143688866.493f75b50f64227c277f1462ad94de11.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-28 03:54:50,006 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 493f75b50f64227c277f1462ad94de11 2023-11-28 03:54:50,006 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 493f75b50f64227c277f1462ad94de11 2023-11-28 03:54:50,007 INFO [StoreOpener-493f75b50f64227c277f1462ad94de11-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family f1 of region 493f75b50f64227c277f1462ad94de11 2023-11-28 03:54:50,009 DEBUG [StoreOpener-493f75b50f64227c277f1462ad94de11-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testGetAfterCompaction/493f75b50f64227c277f1462ad94de11/f1 2023-11-28 03:54:50,009 DEBUG [StoreOpener-493f75b50f64227c277f1462ad94de11-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testGetAfterCompaction/493f75b50f64227c277f1462ad94de11/f1 2023-11-28 03:54:50,009 INFO [StoreOpener-493f75b50f64227c277f1462ad94de11-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 493f75b50f64227c277f1462ad94de11 columnFamilyName f1 2023-11-28 03:54:50,010 INFO [StoreOpener-493f75b50f64227c277f1462ad94de11-1] regionserver.HStore(310): Store=493f75b50f64227c277f1462ad94de11/f1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-28 03:54:50,011 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testGetAfterCompaction/493f75b50f64227c277f1462ad94de11 2023-11-28 03:54:50,011 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testGetAfterCompaction/493f75b50f64227c277f1462ad94de11 2023-11-28 03:54:50,014 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 493f75b50f64227c277f1462ad94de11 2023-11-28 03:54:50,015 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testGetAfterCompaction/493f75b50f64227c277f1462ad94de11/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-11-28 03:54:50,016 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 493f75b50f64227c277f1462ad94de11; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74808609, jitterRate=0.11473514139652252}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-11-28 03:54:50,016 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 493f75b50f64227c277f1462ad94de11: 2023-11-28 03:54:50,017 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2339): Post open deploy tasks for testGetAfterCompaction,,1701143688866.493f75b50f64227c277f1462ad94de11., pid=73, masterSystemTime=1701143690001 2023-11-28 03:54:50,018 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2366): Finished post open deploy task for testGetAfterCompaction,,1701143688866.493f75b50f64227c277f1462ad94de11. 2023-11-28 03:54:50,018 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testGetAfterCompaction,,1701143688866.493f75b50f64227c277f1462ad94de11. 2023-11-28 03:54:50,018 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=72 updating hbase:meta row=493f75b50f64227c277f1462ad94de11, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:50,019 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testGetAfterCompaction,,1701143688866.493f75b50f64227c277f1462ad94de11.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1701143690018"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1701143690018"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1701143690018"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1701143690018"}]},"ts":"1701143690018"} 2023-11-28 03:54:50,021 INFO [PEWorker-2] procedure2.ProcedureExecutor(1825): Finished subprocedure pid=73, resume processing ppid=72 2023-11-28 03:54:50,021 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=73, ppid=72, state=SUCCESS; OpenRegionProcedure 493f75b50f64227c277f1462ad94de11, server=jenkins-hbase4.apache.org,41203,1701143600653 in 171 msec 2023-11-28 03:54:50,022 INFO [PEWorker-4] procedure2.ProcedureExecutor(1825): Finished subprocedure pid=72, resume processing ppid=71 2023-11-28 03:54:50,022 INFO [PEWorker-4] procedure2.ProcedureExecutor(1409): Finished pid=72, ppid=71, state=SUCCESS; TransitRegionStateProcedure table=testGetAfterCompaction, region=493f75b50f64227c277f1462ad94de11, ASSIGN in 326 msec 2023-11-28 03:54:50,023 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-11-28 03:54:50,023 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testGetAfterCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1701143690023"}]},"ts":"1701143690023"} 2023-11-28 03:54:50,024 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testGetAfterCompaction, state=ENABLED in hbase:meta 2023-11-28 03:54:50,026 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_POST_OPERATION 2023-11-28 03:54:50,027 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=71, state=SUCCESS; CreateTableProcedure table=testGetAfterCompaction in 1.1600 sec 2023-11-28 03:54:50,975 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(1230): Checking to see if procedure is done pid=71 2023-11-28 03:54:50,975 INFO [Listener at localhost/42853] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testGetAfterCompaction, procId: 71 completed 2023-11-28 03:54:50,975 DEBUG [Listener at localhost/42853] hbase.HBaseTestingUtility(3430): Waiting until all regions of table testGetAfterCompaction get assigned. Timeout = 60000ms 2023-11-28 03:54:50,977 INFO [Listener at localhost/42853] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-11-28 03:54:50,980 INFO [Listener at localhost/42853] hbase.HBaseTestingUtility(3484): All regions for table testGetAfterCompaction assigned to meta. Checking AM states. 2023-11-28 03:54:50,980 INFO [Listener at localhost/42853] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-11-28 03:54:50,980 INFO [Listener at localhost/42853] hbase.HBaseTestingUtility(3504): All regions for table testGetAfterCompaction assigned. 2023-11-28 03:54:50,984 INFO [Listener at localhost/42853] regionserver.HRegion(2745): Flushing 493f75b50f64227c277f1462ad94de11 1/1 column families, dataSize=37 B heapSize=368 B 2023-11-28 03:54:51,396 INFO [Listener at localhost/42853] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023112811297cd5e8c641f489f2643fa7bb4446 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e2023112811297cd5e8c641f489f2643fa7bb4446 2023-11-28 03:54:51,802 INFO [Listener at localhost/42853] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=37, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testGetAfterCompaction/493f75b50f64227c277f1462ad94de11/.tmp/f1/0d731221a5b3493696fb43ca3a4f344e 2023-11-28 03:54:51,807 DEBUG [Listener at localhost/42853] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testGetAfterCompaction/493f75b50f64227c277f1462ad94de11/.tmp/f1/0d731221a5b3493696fb43ca3a4f344e as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testGetAfterCompaction/493f75b50f64227c277f1462ad94de11/f1/0d731221a5b3493696fb43ca3a4f344e 2023-11-28 03:54:51,811 INFO [Listener at localhost/42853] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testGetAfterCompaction/493f75b50f64227c277f1462ad94de11/f1/0d731221a5b3493696fb43ca3a4f344e, entries=1, sequenceid=5, filesize=4.9 K 2023-11-28 03:54:51,812 INFO [Listener at localhost/42853] regionserver.HRegion(2948): Finished flush of dataSize ~37 B/37, heapSize ~352 B/352, currentSize=0 B/0 for 493f75b50f64227c277f1462ad94de11 in 828ms, sequenceid=5, compaction requested=false 2023-11-28 03:54:51,812 DEBUG [Listener at localhost/42853] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testGetAfterCompaction' 2023-11-28 03:54:51,812 DEBUG [Listener at localhost/42853] regionserver.HRegion(2446): Flush status journal for 493f75b50f64227c277f1462ad94de11: 2023-11-28 03:54:51,814 INFO [Listener at localhost/42853] regionserver.HRegion(2745): Flushing 493f75b50f64227c277f1462ad94de11 1/1 column families, dataSize=37 B heapSize=368 B 2023-11-28 03:54:52,115 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-11-28 03:54:52,225 INFO [Listener at localhost/42853] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20231128529026992ab449f5b10e953c110aec64 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e20231128529026992ab449f5b10e953c110aec64 2023-11-28 03:54:52,630 INFO [Listener at localhost/42853] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=37, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testGetAfterCompaction/493f75b50f64227c277f1462ad94de11/.tmp/f1/bc4772836a754e68bdde72eb6e2eaedb 2023-11-28 03:54:52,635 DEBUG [Listener at localhost/42853] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testGetAfterCompaction/493f75b50f64227c277f1462ad94de11/.tmp/f1/bc4772836a754e68bdde72eb6e2eaedb as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testGetAfterCompaction/493f75b50f64227c277f1462ad94de11/f1/bc4772836a754e68bdde72eb6e2eaedb 2023-11-28 03:54:52,639 INFO [Listener at localhost/42853] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testGetAfterCompaction/493f75b50f64227c277f1462ad94de11/f1/bc4772836a754e68bdde72eb6e2eaedb, entries=1, sequenceid=9, filesize=4.9 K 2023-11-28 03:54:52,639 INFO [Listener at localhost/42853] regionserver.HRegion(2948): Finished flush of dataSize ~37 B/37, heapSize ~352 B/352, currentSize=0 B/0 for 493f75b50f64227c277f1462ad94de11 in 825ms, sequenceid=9, compaction requested=false 2023-11-28 03:54:52,640 DEBUG [Listener at localhost/42853] regionserver.HRegion(2446): Flush status journal for 493f75b50f64227c277f1462ad94de11: 2023-11-28 03:54:52,641 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=38593] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'f1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}] of the table testGetAfterCompaction 2023-11-28 03:54:52,642 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] procedure2.ProcedureExecutor(1028): Stored pid=74, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testGetAfterCompaction.mobLock, type=EXCLUSIVE 2023-11-28 03:54:52,643 DEBUG [PEWorker-3] locking.LockProcedure(309): LOCKED pid=74, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testGetAfterCompaction.mobLock, type=EXCLUSIVE 2023-11-28 03:54:52,643 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] mob.MobUtils(833): start MOB compaction of files for table='testGetAfterCompaction', column='f1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-11-28 03:54:52,643 INFO [PEWorker-3] procedure2.TimeoutExecutorThread(81): ADDED pid=74, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testGetAfterCompaction.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1701144292643 2023-11-28 03:54:52,644 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-11-28 03:54:52,644 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 2 selected files, and 0 irrelevant files table 'testGetAfterCompaction' and column 'f1' 2023-11-28 03:54:52,644 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testGetAfterCompaction' column='f1' 2023-11-28 03:54:52,645 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] zookeeper.ReadOnlyZKClient(139): Connect 0x7ceb165a to 127.0.0.1:55641 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-11-28 03:54:52,803 INFO [Finalizer] client.ConnectionImplementation(1973): Closing master protocol: MasterService 2023-11-28 03:54:52,804 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x21180dfe to 127.0.0.1:55641 2023-11-28 03:54:52,804 DEBUG [Finalizer] ipc.AbstractRpcClient(489): Stopping rpc client 2023-11-28 03:54:52,804 INFO [Finalizer] client.ConnectionImplementation(1973): Closing master protocol: MasterService 2023-11-28 03:54:52,804 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x3f139172 to 127.0.0.1:55641 2023-11-28 03:54:52,804 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(142): Chore: CompactedHFilesCleaner missed its start time 2023-11-28 03:54:52,804 INFO [Listener at localhost/42853] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testGetAfterCompaction/493f75b50f64227c277f1462ad94de11/f1/0d731221a5b3493696fb43ca3a4f344e; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1701143691801; access_time=1701143690987; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-28 03:54:52,805 INFO [Listener at localhost/42853] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testGetAfterCompaction/493f75b50f64227c277f1462ad94de11/f1/bc4772836a754e68bdde72eb6e2eaedb; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1701143692630; access_time=1701143691816; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-28 03:54:52,805 DEBUG [Finalizer] ipc.AbstractRpcClient(489): Stopping rpc client 2023-11-28 03:54:52,806 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x3c94566c to 127.0.0.1:55641 2023-11-28 03:54:52,806 DEBUG [Finalizer] ipc.AbstractRpcClient(489): Stopping rpc client 2023-11-28 03:54:52,813 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] ipc.AbstractRpcClient(189): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@517629d1, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-11-28 03:54:52,813 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20231128 for table='testGetAfterCompaction' and column='f1' 2023-11-28 03:54:52,855 INFO [Listener at localhost/42853] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testGetAfterCompaction/493f75b50f64227c277f1462ad94de11/f1/0d731221a5b3493696fb43ca3a4f344e; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1701143691801; access_time=1701143690987; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-28 03:54:52,856 INFO [Listener at localhost/42853] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testGetAfterCompaction/493f75b50f64227c277f1462ad94de11/f1/bc4772836a754e68bdde72eb6e2eaedb; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1701143692630; access_time=1701143691816; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-28 03:54:52,906 INFO [Listener at localhost/42853] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testGetAfterCompaction/493f75b50f64227c277f1462ad94de11/f1/0d731221a5b3493696fb43ca3a4f344e; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1701143691801; access_time=1701143690987; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-28 03:54:52,906 INFO [Listener at localhost/42853] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testGetAfterCompaction/493f75b50f64227c277f1462ad94de11/f1/bc4772836a754e68bdde72eb6e2eaedb; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1701143692630; access_time=1701143691816; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-28 03:54:52,957 INFO [Listener at localhost/42853] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testGetAfterCompaction/493f75b50f64227c277f1462ad94de11/f1/0d731221a5b3493696fb43ca3a4f344e; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1701143691801; access_time=1701143690987; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-28 03:54:52,957 INFO [Listener at localhost/42853] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testGetAfterCompaction/493f75b50f64227c277f1462ad94de11/f1/bc4772836a754e68bdde72eb6e2eaedb; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1701143692630; access_time=1701143691816; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-28 03:54:53,008 INFO [Listener at localhost/42853] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testGetAfterCompaction/493f75b50f64227c277f1462ad94de11/f1/0d731221a5b3493696fb43ca3a4f344e; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1701143691801; access_time=1701143690987; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-28 03:54:53,008 INFO [Listener at localhost/42853] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testGetAfterCompaction/493f75b50f64227c277f1462ad94de11/f1/bc4772836a754e68bdde72eb6e2eaedb; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1701143692630; access_time=1701143691816; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-28 03:54:53,059 INFO [Listener at localhost/42853] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testGetAfterCompaction/493f75b50f64227c277f1462ad94de11/f1/0d731221a5b3493696fb43ca3a4f344e; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1701143691801; access_time=1701143690987; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-28 03:54:53,059 INFO [Listener at localhost/42853] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testGetAfterCompaction/493f75b50f64227c277f1462ad94de11/f1/bc4772836a754e68bdde72eb6e2eaedb; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1701143692630; access_time=1701143691816; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-28 03:54:53,110 INFO [Listener at localhost/42853] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testGetAfterCompaction/493f75b50f64227c277f1462ad94de11/f1/0d731221a5b3493696fb43ca3a4f344e; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1701143691801; access_time=1701143690987; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-28 03:54:53,110 INFO [Listener at localhost/42853] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testGetAfterCompaction/493f75b50f64227c277f1462ad94de11/f1/bc4772836a754e68bdde72eb6e2eaedb; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1701143692630; access_time=1701143691816; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-28 03:54:53,161 INFO [Listener at localhost/42853] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testGetAfterCompaction/493f75b50f64227c277f1462ad94de11/f1/0d731221a5b3493696fb43ca3a4f344e; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1701143691801; access_time=1701143690987; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-28 03:54:53,161 INFO [Listener at localhost/42853] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testGetAfterCompaction/493f75b50f64227c277f1462ad94de11/f1/bc4772836a754e68bdde72eb6e2eaedb; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1701143692630; access_time=1701143691816; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-28 03:54:53,211 INFO [Listener at localhost/42853] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testGetAfterCompaction/493f75b50f64227c277f1462ad94de11/f1/0d731221a5b3493696fb43ca3a4f344e; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1701143691801; access_time=1701143690987; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-28 03:54:53,212 INFO [Listener at localhost/42853] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testGetAfterCompaction/493f75b50f64227c277f1462ad94de11/f1/bc4772836a754e68bdde72eb6e2eaedb; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1701143692630; access_time=1701143691816; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-28 03:54:53,262 INFO [Listener at localhost/42853] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testGetAfterCompaction/493f75b50f64227c277f1462ad94de11/f1/0d731221a5b3493696fb43ca3a4f344e; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1701143691801; access_time=1701143690987; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-28 03:54:53,262 INFO [Listener at localhost/42853] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testGetAfterCompaction/493f75b50f64227c277f1462ad94de11/f1/bc4772836a754e68bdde72eb6e2eaedb; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1701143692630; access_time=1701143691816; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-28 03:54:53,313 INFO [Listener at localhost/42853] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testGetAfterCompaction/493f75b50f64227c277f1462ad94de11/f1/0d731221a5b3493696fb43ca3a4f344e; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1701143691801; access_time=1701143690987; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-28 03:54:53,313 INFO [Listener at localhost/42853] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testGetAfterCompaction/493f75b50f64227c277f1462ad94de11/f1/bc4772836a754e68bdde72eb6e2eaedb; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1701143692630; access_time=1701143691816; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-28 03:54:53,364 INFO [Listener at localhost/42853] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testGetAfterCompaction/493f75b50f64227c277f1462ad94de11/f1/0d731221a5b3493696fb43ca3a4f344e; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1701143691801; access_time=1701143690987; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-28 03:54:53,364 INFO [Listener at localhost/42853] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testGetAfterCompaction/493f75b50f64227c277f1462ad94de11/f1/bc4772836a754e68bdde72eb6e2eaedb; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1701143692630; access_time=1701143691816; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-28 03:54:53,415 INFO [Listener at localhost/42853] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testGetAfterCompaction/493f75b50f64227c277f1462ad94de11/f1/0d731221a5b3493696fb43ca3a4f344e; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1701143691801; access_time=1701143690987; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-28 03:54:53,415 INFO [Listener at localhost/42853] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testGetAfterCompaction/493f75b50f64227c277f1462ad94de11/f1/bc4772836a754e68bdde72eb6e2eaedb; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1701143692630; access_time=1701143691816; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-28 03:54:53,466 INFO [Listener at localhost/42853] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testGetAfterCompaction/493f75b50f64227c277f1462ad94de11/f1/0d731221a5b3493696fb43ca3a4f344e; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1701143691801; access_time=1701143690987; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-28 03:54:53,466 INFO [Listener at localhost/42853] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testGetAfterCompaction/493f75b50f64227c277f1462ad94de11/f1/bc4772836a754e68bdde72eb6e2eaedb; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1701143692630; access_time=1701143691816; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-28 03:54:53,517 INFO [Listener at localhost/42853] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testGetAfterCompaction/493f75b50f64227c277f1462ad94de11/f1/0d731221a5b3493696fb43ca3a4f344e; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1701143691801; access_time=1701143690987; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-28 03:54:53,517 INFO [Listener at localhost/42853] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testGetAfterCompaction/493f75b50f64227c277f1462ad94de11/f1/bc4772836a754e68bdde72eb6e2eaedb; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1701143692630; access_time=1701143691816; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-28 03:54:53,568 INFO [Listener at localhost/42853] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testGetAfterCompaction/493f75b50f64227c277f1462ad94de11/f1/0d731221a5b3493696fb43ca3a4f344e; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1701143691801; access_time=1701143690987; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-28 03:54:53,568 INFO [Listener at localhost/42853] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testGetAfterCompaction/493f75b50f64227c277f1462ad94de11/f1/bc4772836a754e68bdde72eb6e2eaedb; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1701143692630; access_time=1701143691816; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-28 03:54:53,619 INFO [Listener at localhost/42853] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testGetAfterCompaction/493f75b50f64227c277f1462ad94de11/f1/0d731221a5b3493696fb43ca3a4f344e; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1701143691801; access_time=1701143690987; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-28 03:54:53,619 INFO [Listener at localhost/42853] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testGetAfterCompaction/493f75b50f64227c277f1462ad94de11/f1/bc4772836a754e68bdde72eb6e2eaedb; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1701143692630; access_time=1701143691816; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-28 03:54:53,639 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202311282ef74dc9675f4aaa91ba34d722ad5d26 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e202311282ef74dc9675f4aaa91ba34d722ad5d26 2023-11-28 03:54:53,640 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20231128 table='testGetAfterCompaction' column='f1' 2023-11-28 03:54:53,659 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(122): Using SIMPLE authentication for service=ClientService, sasl=false 2023-11-28 03:54:53,660 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:51606, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-11-28 03:54:53,667 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(122): Using SIMPLE authentication for service=MasterService, sasl=false 2023-11-28 03:54:53,669 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:38778, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-11-28 03:54:53,669 INFO [Listener at localhost/42853] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testGetAfterCompaction/493f75b50f64227c277f1462ad94de11/f1/0d731221a5b3493696fb43ca3a4f344e; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1701143691801; access_time=1701143690987; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-28 03:54:53,670 INFO [Listener at localhost/42853] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testGetAfterCompaction/493f75b50f64227c277f1462ad94de11/f1/bc4772836a754e68bdde72eb6e2eaedb; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1701143692630; access_time=1701143691816; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-28 03:54:53,679 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testGetAfterCompaction/d41d8cd98f00b204e9800998ecf8427e20231128/f1/59d7bf43f68c481e80338c3d58564ebb first=Optional[r1] last=Optional[r2] 2023-11-28 03:54:53,680 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testGetAfterCompaction,,1701143688866.493f75b50f64227c277f1462ad94de11., hostname=jenkins-hbase4.apache.org,41203,1701143600653, seqNum=2 for row with hfile group [{f1,hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testGetAfterCompaction/d41d8cd98f00b204e9800998ecf8427e20231128/f1/59d7bf43f68c481e80338c3d58564ebb}] 2023-11-28 03:54:53,684 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HStore(763): Validating hfile at hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testGetAfterCompaction/d41d8cd98f00b204e9800998ecf8427e20231128/f1/59d7bf43f68c481e80338c3d58564ebb for inclusion in 493f75b50f64227c277f1462ad94de11/f1 2023-11-28 03:54:53,687 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HStore(775): HFile bounds: first=r1 last=r2 2023-11-28 03:54:53,687 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HStore(777): Region bounds: first= last= 2023-11-28 03:54:53,687 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HRegion(2446): Flush status journal for 493f75b50f64227c277f1462ad94de11: 2023-11-28 03:54:53,688 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testGetAfterCompaction/d41d8cd98f00b204e9800998ecf8427e20231128/f1/59d7bf43f68c481e80338c3d58564ebb to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testGetAfterCompaction__cqvdq3ndi9po6gdge559gekon965p2c0tjrsr6d5t7ei3rfvtlj538shqrg4hm93/f1/59d7bf43f68c481e80338c3d58564ebb 2023-11-28 03:54:53,690 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testGetAfterCompaction__cqvdq3ndi9po6gdge559gekon965p2c0tjrsr6d5t7ei3rfvtlj538shqrg4hm93/f1/59d7bf43f68c481e80338c3d58564ebb as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testGetAfterCompaction/493f75b50f64227c277f1462ad94de11/f1/da21eb52d49d4b8593c422f19b52bdb0_SeqId_12_ 2023-11-28 03:54:53,691 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HStore(849): Loaded HFile hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testGetAfterCompaction__cqvdq3ndi9po6gdge559gekon965p2c0tjrsr6d5t7ei3rfvtlj538shqrg4hm93/f1/59d7bf43f68c481e80338c3d58564ebb into 493f75b50f64227c277f1462ad94de11/f1 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testGetAfterCompaction/493f75b50f64227c277f1462ad94de11/f1/da21eb52d49d4b8593c422f19b52bdb0_SeqId_12_ - updating store file list. 2023-11-28 03:54:53,694 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HStore(882): Loaded HFile hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testGetAfterCompaction/493f75b50f64227c277f1462ad94de11/f1/da21eb52d49d4b8593c422f19b52bdb0_SeqId_12_ into 493f75b50f64227c277f1462ad94de11/f1 2023-11-28 03:54:53,694 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HStore(855): Successfully loaded hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testGetAfterCompaction__cqvdq3ndi9po6gdge559gekon965p2c0tjrsr6d5t7ei3rfvtlj538shqrg4hm93/f1/59d7bf43f68c481e80338c3d58564ebb into 493f75b50f64227c277f1462ad94de11/f1 (new location: hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testGetAfterCompaction/493f75b50f64227c277f1462ad94de11/f1/da21eb52d49d4b8593c422f19b52bdb0_SeqId_12_) 2023-11-28 03:54:53,695 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testGetAfterCompaction__cqvdq3ndi9po6gdge559gekon965p2c0tjrsr6d5t7ei3rfvtlj538shqrg4hm93/f1/59d7bf43f68c481e80338c3d58564ebb 2023-11-28 03:54:53,695 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HStore(1912): 493f75b50f64227c277f1462ad94de11/f1 is initiating minor compaction 2023-11-28 03:54:53,695 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@52263b9; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-28 03:54:53,695 INFO [RS:0;jenkins-hbase4:41203-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 493f75b50f64227c277f1462ad94de11/f1 in testGetAfterCompaction,,1701143688866.493f75b50f64227c277f1462ad94de11. 2023-11-28 03:54:53,696 INFO [RS:0;jenkins-hbase4:41203-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testGetAfterCompaction/493f75b50f64227c277f1462ad94de11/f1/bc4772836a754e68bdde72eb6e2eaedb, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testGetAfterCompaction/493f75b50f64227c277f1462ad94de11/f1/da21eb52d49d4b8593c422f19b52bdb0_SeqId_12_] into tmpdir=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testGetAfterCompaction/493f75b50f64227c277f1462ad94de11/.tmp, totalSize=10.0 K 2023-11-28 03:54:53,696 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] compactions.Compactor(207): Compacting bc4772836a754e68bdde72eb6e2eaedb, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=9 2023-11-28 03:54:53,696 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] compactions.Compactor(207): Compacting da21eb52d49d4b8593c422f19b52bdb0_SeqId_12_, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13 2023-11-28 03:54:53,698 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20231128 table='testGetAfterCompaction' column='f1' 2023-11-28 03:54:53,698 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-28 03:54:53,700 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e2023112811297cd5e8c641f489f2643fa7bb4446 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e2023112811297cd5e8c641f489f2643fa7bb4446 2023-11-28 03:54:53,701 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e20231128529026992ab449f5b10e953c110aec64 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e20231128529026992ab449f5b10e953c110aec64 2023-11-28 03:54:53,702 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testGetAfterCompaction 2 expired mob files are deleted 2023-11-28 03:54:53,702 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testGetAfterCompaction/d41d8cd98f00b204e9800998ecf8427e20231128' 2023-11-28 03:54:53,702 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20231128 for table='testGetAfterCompaction' and column='f1' 2023-11-28 03:54:53,702 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] compactions.PartitionedMobCompactor(354): After compaction, there are 1 mob files. table='testGetAfterCompaction' column='f1' 2023-11-28 03:54:53,702 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] mob.MobUtils(840): end MOB compaction of files for table='testGetAfterCompaction', column='f1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-11-28 03:54:53,706 DEBUG [PEWorker-1] locking.LockProcedure(242): UNLOCKED pid=74, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testGetAfterCompaction.mobLock, type=EXCLUSIVE 2023-11-28 03:54:53,708 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=74, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testGetAfterCompaction.mobLock, type=EXCLUSIVE in 1.0660 sec 2023-11-28 03:54:53,708 INFO [RS:0;jenkins-hbase4:41203-shortCompactions-0] throttle.PressureAwareThroughputController(145): 493f75b50f64227c277f1462ad94de11#f1#compaction#76 average throughput is 0.25 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-11-28 03:54:53,720 INFO [Listener at localhost/42853] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testGetAfterCompaction/493f75b50f64227c277f1462ad94de11/f1/0d731221a5b3493696fb43ca3a4f344e; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1701143691801; access_time=1701143690987; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-28 03:54:53,720 INFO [Listener at localhost/42853] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testGetAfterCompaction/493f75b50f64227c277f1462ad94de11/f1/bc4772836a754e68bdde72eb6e2eaedb; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1701143692630; access_time=1701143691816; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-28 03:54:53,720 INFO [Listener at localhost/42853] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testGetAfterCompaction/493f75b50f64227c277f1462ad94de11/f1/da21eb52d49d4b8593c422f19b52bdb0_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1701143693635; access_time=1701143692825; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-11-28 03:54:53,724 DEBUG [Listener at localhost/42853] ipc.RpcConnection(122): Using SIMPLE authentication for service=AdminService, sasl=false 2023-11-28 03:54:53,727 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:51616, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-11-28 03:54:53,727 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=41203] regionserver.RSRpcServices(1662): Compacting testGetAfterCompaction,,1701143688866.493f75b50f64227c277f1462ad94de11. 2023-11-28 03:54:53,727 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=41203] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 2 compacting, 0 eligible, 16 blocking 2023-11-28 03:54:53,728 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=41203] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 0 files of size 0 starting at candidate #-1 after considering 0 permutations with 0 in ratio 2023-11-28 03:54:53,728 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=41203] compactions.SortedCompactionPolicy(232): Not compacting files because we only have 0 files ready for compaction. Need 3 to initiate. 2023-11-28 03:54:53,728 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=41203] regionserver.CompactSplit(415): Not compacting testGetAfterCompaction,,1701143688866.493f75b50f64227c277f1462ad94de11. because compaction request was cancelled 2023-11-28 03:54:53,779 INFO [Listener at localhost/42853] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testGetAfterCompaction/493f75b50f64227c277f1462ad94de11/f1/0d731221a5b3493696fb43ca3a4f344e; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1701143691801; access_time=1701143690987; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-28 03:54:53,779 INFO [Listener at localhost/42853] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testGetAfterCompaction/493f75b50f64227c277f1462ad94de11/f1/bc4772836a754e68bdde72eb6e2eaedb; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1701143692630; access_time=1701143691816; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-28 03:54:53,779 INFO [Listener at localhost/42853] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testGetAfterCompaction/493f75b50f64227c277f1462ad94de11/f1/da21eb52d49d4b8593c422f19b52bdb0_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1701143693635; access_time=1701143692825; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-11-28 03:54:53,830 INFO [Listener at localhost/42853] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testGetAfterCompaction/493f75b50f64227c277f1462ad94de11/f1/0d731221a5b3493696fb43ca3a4f344e; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1701143691801; access_time=1701143690987; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-28 03:54:53,830 INFO [Listener at localhost/42853] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testGetAfterCompaction/493f75b50f64227c277f1462ad94de11/f1/bc4772836a754e68bdde72eb6e2eaedb; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1701143692630; access_time=1701143691816; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-28 03:54:53,830 INFO [Listener at localhost/42853] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testGetAfterCompaction/493f75b50f64227c277f1462ad94de11/f1/da21eb52d49d4b8593c422f19b52bdb0_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1701143693635; access_time=1701143692825; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-11-28 03:54:53,881 INFO [Listener at localhost/42853] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testGetAfterCompaction/493f75b50f64227c277f1462ad94de11/f1/0d731221a5b3493696fb43ca3a4f344e; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1701143691801; access_time=1701143690987; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-28 03:54:53,881 INFO [Listener at localhost/42853] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testGetAfterCompaction/493f75b50f64227c277f1462ad94de11/f1/bc4772836a754e68bdde72eb6e2eaedb; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1701143692630; access_time=1701143691816; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-28 03:54:53,881 INFO [Listener at localhost/42853] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testGetAfterCompaction/493f75b50f64227c277f1462ad94de11/f1/da21eb52d49d4b8593c422f19b52bdb0_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1701143693635; access_time=1701143692825; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-11-28 03:54:53,932 INFO [Listener at localhost/42853] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testGetAfterCompaction/493f75b50f64227c277f1462ad94de11/f1/0d731221a5b3493696fb43ca3a4f344e; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1701143691801; access_time=1701143690987; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-28 03:54:53,932 INFO [Listener at localhost/42853] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testGetAfterCompaction/493f75b50f64227c277f1462ad94de11/f1/bc4772836a754e68bdde72eb6e2eaedb; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1701143692630; access_time=1701143691816; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-28 03:54:53,932 INFO [Listener at localhost/42853] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testGetAfterCompaction/493f75b50f64227c277f1462ad94de11/f1/da21eb52d49d4b8593c422f19b52bdb0_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1701143693635; access_time=1701143692825; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-11-28 03:54:53,983 INFO [Listener at localhost/42853] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testGetAfterCompaction/493f75b50f64227c277f1462ad94de11/f1/0d731221a5b3493696fb43ca3a4f344e; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1701143691801; access_time=1701143690987; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-28 03:54:53,983 INFO [Listener at localhost/42853] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testGetAfterCompaction/493f75b50f64227c277f1462ad94de11/f1/bc4772836a754e68bdde72eb6e2eaedb; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1701143692630; access_time=1701143691816; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-28 03:54:53,983 INFO [Listener at localhost/42853] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testGetAfterCompaction/493f75b50f64227c277f1462ad94de11/f1/da21eb52d49d4b8593c422f19b52bdb0_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1701143693635; access_time=1701143692825; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-11-28 03:54:54,034 INFO [Listener at localhost/42853] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testGetAfterCompaction/493f75b50f64227c277f1462ad94de11/f1/0d731221a5b3493696fb43ca3a4f344e; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1701143691801; access_time=1701143690987; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-28 03:54:54,034 INFO [Listener at localhost/42853] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testGetAfterCompaction/493f75b50f64227c277f1462ad94de11/f1/bc4772836a754e68bdde72eb6e2eaedb; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1701143692630; access_time=1701143691816; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-28 03:54:54,034 INFO [Listener at localhost/42853] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testGetAfterCompaction/493f75b50f64227c277f1462ad94de11/f1/da21eb52d49d4b8593c422f19b52bdb0_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1701143693635; access_time=1701143692825; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-11-28 03:54:54,085 INFO [Listener at localhost/42853] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testGetAfterCompaction/493f75b50f64227c277f1462ad94de11/f1/0d731221a5b3493696fb43ca3a4f344e; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1701143691801; access_time=1701143690987; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-28 03:54:54,085 INFO [Listener at localhost/42853] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testGetAfterCompaction/493f75b50f64227c277f1462ad94de11/f1/bc4772836a754e68bdde72eb6e2eaedb; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1701143692630; access_time=1701143691816; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-28 03:54:54,085 INFO [Listener at localhost/42853] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testGetAfterCompaction/493f75b50f64227c277f1462ad94de11/f1/da21eb52d49d4b8593c422f19b52bdb0_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1701143693635; access_time=1701143692825; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-11-28 03:54:54,135 INFO [Listener at localhost/42853] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testGetAfterCompaction/493f75b50f64227c277f1462ad94de11/f1/0d731221a5b3493696fb43ca3a4f344e; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1701143691801; access_time=1701143690987; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-28 03:54:54,136 INFO [Listener at localhost/42853] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testGetAfterCompaction/493f75b50f64227c277f1462ad94de11/f1/bc4772836a754e68bdde72eb6e2eaedb; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1701143692630; access_time=1701143691816; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-28 03:54:54,136 INFO [Listener at localhost/42853] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testGetAfterCompaction/493f75b50f64227c277f1462ad94de11/f1/da21eb52d49d4b8593c422f19b52bdb0_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1701143693635; access_time=1701143692825; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-11-28 03:54:54,186 INFO [Listener at localhost/42853] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testGetAfterCompaction/493f75b50f64227c277f1462ad94de11/f1/0d731221a5b3493696fb43ca3a4f344e; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1701143691801; access_time=1701143690987; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-28 03:54:54,187 INFO [Listener at localhost/42853] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testGetAfterCompaction/493f75b50f64227c277f1462ad94de11/f1/bc4772836a754e68bdde72eb6e2eaedb; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1701143692630; access_time=1701143691816; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-28 03:54:54,187 INFO [Listener at localhost/42853] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testGetAfterCompaction/493f75b50f64227c277f1462ad94de11/f1/da21eb52d49d4b8593c422f19b52bdb0_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1701143693635; access_time=1701143692825; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-11-28 03:54:54,237 INFO [Listener at localhost/42853] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testGetAfterCompaction/493f75b50f64227c277f1462ad94de11/f1/0d731221a5b3493696fb43ca3a4f344e; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1701143691801; access_time=1701143690987; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-28 03:54:54,237 INFO [Listener at localhost/42853] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testGetAfterCompaction/493f75b50f64227c277f1462ad94de11/f1/bc4772836a754e68bdde72eb6e2eaedb; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1701143692630; access_time=1701143691816; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-28 03:54:54,237 INFO [Listener at localhost/42853] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testGetAfterCompaction/493f75b50f64227c277f1462ad94de11/f1/da21eb52d49d4b8593c422f19b52bdb0_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1701143693635; access_time=1701143692825; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-11-28 03:54:54,288 INFO [Listener at localhost/42853] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testGetAfterCompaction/493f75b50f64227c277f1462ad94de11/f1/0d731221a5b3493696fb43ca3a4f344e; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1701143691801; access_time=1701143690987; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-28 03:54:54,288 INFO [Listener at localhost/42853] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testGetAfterCompaction/493f75b50f64227c277f1462ad94de11/f1/bc4772836a754e68bdde72eb6e2eaedb; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1701143692630; access_time=1701143691816; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-28 03:54:54,288 INFO [Listener at localhost/42853] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testGetAfterCompaction/493f75b50f64227c277f1462ad94de11/f1/da21eb52d49d4b8593c422f19b52bdb0_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1701143693635; access_time=1701143692825; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-11-28 03:54:54,339 INFO [Listener at localhost/42853] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testGetAfterCompaction/493f75b50f64227c277f1462ad94de11/f1/0d731221a5b3493696fb43ca3a4f344e; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1701143691801; access_time=1701143690987; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-28 03:54:54,339 INFO [Listener at localhost/42853] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testGetAfterCompaction/493f75b50f64227c277f1462ad94de11/f1/bc4772836a754e68bdde72eb6e2eaedb; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1701143692630; access_time=1701143691816; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-28 03:54:54,339 INFO [Listener at localhost/42853] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testGetAfterCompaction/493f75b50f64227c277f1462ad94de11/f1/da21eb52d49d4b8593c422f19b52bdb0_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1701143693635; access_time=1701143692825; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-11-28 03:54:54,390 INFO [Listener at localhost/42853] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testGetAfterCompaction/493f75b50f64227c277f1462ad94de11/f1/0d731221a5b3493696fb43ca3a4f344e; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1701143691801; access_time=1701143690987; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-28 03:54:54,390 INFO [Listener at localhost/42853] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testGetAfterCompaction/493f75b50f64227c277f1462ad94de11/f1/bc4772836a754e68bdde72eb6e2eaedb; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1701143692630; access_time=1701143691816; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-28 03:54:54,390 INFO [Listener at localhost/42853] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testGetAfterCompaction/493f75b50f64227c277f1462ad94de11/f1/da21eb52d49d4b8593c422f19b52bdb0_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1701143693635; access_time=1701143692825; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-11-28 03:54:54,441 INFO [Listener at localhost/42853] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testGetAfterCompaction/493f75b50f64227c277f1462ad94de11/f1/0d731221a5b3493696fb43ca3a4f344e; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1701143691801; access_time=1701143690987; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-28 03:54:54,441 INFO [Listener at localhost/42853] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testGetAfterCompaction/493f75b50f64227c277f1462ad94de11/f1/bc4772836a754e68bdde72eb6e2eaedb; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1701143692630; access_time=1701143691816; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-28 03:54:54,441 INFO [Listener at localhost/42853] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testGetAfterCompaction/493f75b50f64227c277f1462ad94de11/f1/da21eb52d49d4b8593c422f19b52bdb0_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1701143693635; access_time=1701143692825; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-11-28 03:54:54,492 INFO [Listener at localhost/42853] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testGetAfterCompaction/493f75b50f64227c277f1462ad94de11/f1/0d731221a5b3493696fb43ca3a4f344e; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1701143691801; access_time=1701143690987; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-28 03:54:54,492 INFO [Listener at localhost/42853] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testGetAfterCompaction/493f75b50f64227c277f1462ad94de11/f1/bc4772836a754e68bdde72eb6e2eaedb; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1701143692630; access_time=1701143691816; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-28 03:54:54,492 INFO [Listener at localhost/42853] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testGetAfterCompaction/493f75b50f64227c277f1462ad94de11/f1/da21eb52d49d4b8593c422f19b52bdb0_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1701143693635; access_time=1701143692825; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-11-28 03:54:54,522 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testGetAfterCompaction/493f75b50f64227c277f1462ad94de11/.tmp/f1/57372a2f96a046049f5f654a377af502 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testGetAfterCompaction/493f75b50f64227c277f1462ad94de11/f1/57372a2f96a046049f5f654a377af502 2023-11-28 03:54:54,527 INFO [RS:0;jenkins-hbase4:41203-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 2 file(s) in 493f75b50f64227c277f1462ad94de11/f1 of 493f75b50f64227c277f1462ad94de11 into 57372a2f96a046049f5f654a377af502(size=5.1 K), total size for store is 10.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-11-28 03:54:54,528 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 493f75b50f64227c277f1462ad94de11: 2023-11-28 03:54:54,528 INFO [RS:0;jenkins-hbase4:41203-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testGetAfterCompaction,,1701143688866.493f75b50f64227c277f1462ad94de11., storeName=f1, fileCount=2, fileSize=10.0 K (4.9 K, 5.1 K), priority=2, time=1701143693695; duration=0sec 2023-11-28 03:54:54,528 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-28 03:54:54,543 INFO [Listener at localhost/42853] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testGetAfterCompaction/493f75b50f64227c277f1462ad94de11/f1/0d731221a5b3493696fb43ca3a4f344e; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1701143691801; access_time=1701143690987; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-28 03:54:54,543 INFO [Listener at localhost/42853] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testGetAfterCompaction/493f75b50f64227c277f1462ad94de11/f1/57372a2f96a046049f5f654a377af502; isDirectory=false; length=5228; replication=1; blocksize=134217728; modification_time=1701143694517; access_time=1701143693701; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-28 03:54:54,543 INFO [Listener at localhost/42853] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testGetAfterCompaction/493f75b50f64227c277f1462ad94de11/f1/bc4772836a754e68bdde72eb6e2eaedb; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1701143692630; access_time=1701143691816; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-28 03:54:54,543 INFO [Listener at localhost/42853] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testGetAfterCompaction/493f75b50f64227c277f1462ad94de11/f1/da21eb52d49d4b8593c422f19b52bdb0_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1701143693635; access_time=1701143692825; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-11-28 03:54:54,594 INFO [Listener at localhost/42853] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testGetAfterCompaction/493f75b50f64227c277f1462ad94de11/f1/0d731221a5b3493696fb43ca3a4f344e; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1701143691801; access_time=1701143690987; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-28 03:54:54,594 INFO [Listener at localhost/42853] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testGetAfterCompaction/493f75b50f64227c277f1462ad94de11/f1/57372a2f96a046049f5f654a377af502; isDirectory=false; length=5228; replication=1; blocksize=134217728; modification_time=1701143694517; access_time=1701143693701; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-28 03:54:54,594 INFO [Listener at localhost/42853] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testGetAfterCompaction/493f75b50f64227c277f1462ad94de11/f1/bc4772836a754e68bdde72eb6e2eaedb; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1701143692630; access_time=1701143691816; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-28 03:54:54,594 INFO [Listener at localhost/42853] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testGetAfterCompaction/493f75b50f64227c277f1462ad94de11/f1/da21eb52d49d4b8593c422f19b52bdb0_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1701143693635; access_time=1701143692825; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-11-28 03:54:54,614 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testGetAfterCompaction/493f75b50f64227c277f1462ad94de11/f1/bc4772836a754e68bdde72eb6e2eaedb, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testGetAfterCompaction/493f75b50f64227c277f1462ad94de11/f1/da21eb52d49d4b8593c422f19b52bdb0_SeqId_12_] to archive 2023-11-28 03:54:54,615 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-28 03:54:54,616 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testGetAfterCompaction/493f75b50f64227c277f1462ad94de11/f1/bc4772836a754e68bdde72eb6e2eaedb to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testGetAfterCompaction/493f75b50f64227c277f1462ad94de11/f1/bc4772836a754e68bdde72eb6e2eaedb 2023-11-28 03:54:54,618 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testGetAfterCompaction/493f75b50f64227c277f1462ad94de11/f1/da21eb52d49d4b8593c422f19b52bdb0_SeqId_12_ to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testGetAfterCompaction/493f75b50f64227c277f1462ad94de11/f1/da21eb52d49d4b8593c422f19b52bdb0_SeqId_12_ 2023-11-28 03:54:54,645 INFO [Listener at localhost/42853] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testGetAfterCompaction/493f75b50f64227c277f1462ad94de11/f1/0d731221a5b3493696fb43ca3a4f344e; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1701143691801; access_time=1701143690987; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-28 03:54:54,645 INFO [Listener at localhost/42853] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testGetAfterCompaction/493f75b50f64227c277f1462ad94de11/f1/57372a2f96a046049f5f654a377af502; isDirectory=false; length=5228; replication=1; blocksize=134217728; modification_time=1701143694517; access_time=1701143693701; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-28 03:54:54,765 INFO [Listener at localhost/42853] client.HBaseAdmin$15(890): Started disable of testGetAfterCompaction 2023-11-28 03:54:54,770 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.HMaster$11(2418): Client=jenkins//172.31.14.131 disable testGetAfterCompaction 2023-11-28 03:54:54,776 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] procedure2.ProcedureExecutor(1028): Stored pid=75, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testGetAfterCompaction 2023-11-28 03:54:54,779 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testGetAfterCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1701143694779"}]},"ts":"1701143694779"} 2023-11-28 03:54:54,780 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testGetAfterCompaction, state=DISABLING in hbase:meta 2023-11-28 03:54:54,781 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(1230): Checking to see if procedure is done pid=75 2023-11-28 03:54:54,783 INFO [PEWorker-2] procedure.DisableTableProcedure(293): Set testGetAfterCompaction to state=DISABLING 2023-11-28 03:54:54,784 INFO [PEWorker-2] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=76, ppid=75, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testGetAfterCompaction, region=493f75b50f64227c277f1462ad94de11, UNASSIGN}] 2023-11-28 03:54:54,786 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=76, ppid=75, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testGetAfterCompaction, region=493f75b50f64227c277f1462ad94de11, UNASSIGN 2023-11-28 03:54:54,786 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=76 updating hbase:meta row=493f75b50f64227c277f1462ad94de11, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:54,786 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testGetAfterCompaction,,1701143688866.493f75b50f64227c277f1462ad94de11.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1701143694786"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1701143694786"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1701143694786"}]},"ts":"1701143694786"} 2023-11-28 03:54:54,788 INFO [PEWorker-4] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=77, ppid=76, state=RUNNABLE; CloseRegionProcedure 493f75b50f64227c277f1462ad94de11, server=jenkins-hbase4.apache.org,41203,1701143600653}] 2023-11-28 03:54:54,882 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(1230): Checking to see if procedure is done pid=75 2023-11-28 03:54:54,939 DEBUG [RSProcedureDispatcher-pool-2] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:54,940 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 493f75b50f64227c277f1462ad94de11 2023-11-28 03:54:54,940 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 493f75b50f64227c277f1462ad94de11, disabling compactions & flushes 2023-11-28 03:54:54,940 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testGetAfterCompaction,,1701143688866.493f75b50f64227c277f1462ad94de11. 2023-11-28 03:54:54,940 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testGetAfterCompaction,,1701143688866.493f75b50f64227c277f1462ad94de11. 2023-11-28 03:54:54,940 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testGetAfterCompaction,,1701143688866.493f75b50f64227c277f1462ad94de11. after waiting 0 ms 2023-11-28 03:54:54,940 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testGetAfterCompaction,,1701143688866.493f75b50f64227c277f1462ad94de11. 2023-11-28 03:54:54,944 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testGetAfterCompaction/493f75b50f64227c277f1462ad94de11/recovered.edits/15.seqid, newMaxSeqId=15, maxSeqId=1 2023-11-28 03:54:54,945 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro 2023-11-28 03:54:54,947 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testGetAfterCompaction,,1701143688866.493f75b50f64227c277f1462ad94de11. 2023-11-28 03:54:54,947 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 493f75b50f64227c277f1462ad94de11: 2023-11-28 03:54:54,948 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 493f75b50f64227c277f1462ad94de11 2023-11-28 03:54:54,948 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=76 updating hbase:meta row=493f75b50f64227c277f1462ad94de11, regionState=CLOSED 2023-11-28 03:54:54,949 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testGetAfterCompaction,,1701143688866.493f75b50f64227c277f1462ad94de11.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1701143694948"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1701143694948"}]},"ts":"1701143694948"} 2023-11-28 03:54:54,951 INFO [PEWorker-3] procedure2.ProcedureExecutor(1825): Finished subprocedure pid=77, resume processing ppid=76 2023-11-28 03:54:54,951 INFO [PEWorker-3] procedure2.ProcedureExecutor(1409): Finished pid=77, ppid=76, state=SUCCESS; CloseRegionProcedure 493f75b50f64227c277f1462ad94de11, server=jenkins-hbase4.apache.org,41203,1701143600653 in 162 msec 2023-11-28 03:54:54,952 INFO [PEWorker-1] procedure2.ProcedureExecutor(1825): Finished subprocedure pid=76, resume processing ppid=75 2023-11-28 03:54:54,952 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=76, ppid=75, state=SUCCESS; TransitRegionStateProcedure table=testGetAfterCompaction, region=493f75b50f64227c277f1462ad94de11, UNASSIGN in 167 msec 2023-11-28 03:54:54,953 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testGetAfterCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1701143694953"}]},"ts":"1701143694953"} 2023-11-28 03:54:54,954 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testGetAfterCompaction, state=DISABLED in hbase:meta 2023-11-28 03:54:54,955 INFO [PEWorker-2] procedure.DisableTableProcedure(305): Set testGetAfterCompaction to state=DISABLED 2023-11-28 03:54:54,957 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=75, state=SUCCESS; DisableTableProcedure table=testGetAfterCompaction in 184 msec 2023-11-28 03:54:55,083 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(1230): Checking to see if procedure is done pid=75 2023-11-28 03:54:55,083 INFO [Listener at localhost/42853] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testGetAfterCompaction, procId: 75 completed 2023-11-28 03:54:55,088 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.HMaster$5(2228): Client=jenkins//172.31.14.131 delete testGetAfterCompaction 2023-11-28 03:54:55,097 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] procedure2.ProcedureExecutor(1028): Stored pid=78, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testGetAfterCompaction 2023-11-28 03:54:55,098 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=78, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testGetAfterCompaction 2023-11-28 03:54:55,100 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=78, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testGetAfterCompaction 2023-11-28 03:54:55,101 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(1230): Checking to see if procedure is done pid=78 2023-11-28 03:54:55,102 DEBUG [HFileArchiver-12] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/.tmp/data/default/testGetAfterCompaction/493f75b50f64227c277f1462ad94de11 2023-11-28 03:54:55,105 DEBUG [HFileArchiver-12] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/.tmp/data/default/testGetAfterCompaction/493f75b50f64227c277f1462ad94de11/f1, FileablePath, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/.tmp/data/default/testGetAfterCompaction/493f75b50f64227c277f1462ad94de11/recovered.edits] 2023-11-28 03:54:55,108 DEBUG [HFileArchiver-12] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/.tmp/data/default/testGetAfterCompaction/493f75b50f64227c277f1462ad94de11/f1/0d731221a5b3493696fb43ca3a4f344e to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testGetAfterCompaction/493f75b50f64227c277f1462ad94de11/f1/0d731221a5b3493696fb43ca3a4f344e 2023-11-28 03:54:55,109 DEBUG [HFileArchiver-12] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/.tmp/data/default/testGetAfterCompaction/493f75b50f64227c277f1462ad94de11/f1/57372a2f96a046049f5f654a377af502 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testGetAfterCompaction/493f75b50f64227c277f1462ad94de11/f1/57372a2f96a046049f5f654a377af502 2023-11-28 03:54:55,112 DEBUG [HFileArchiver-12] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/.tmp/data/default/testGetAfterCompaction/493f75b50f64227c277f1462ad94de11/recovered.edits/15.seqid to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testGetAfterCompaction/493f75b50f64227c277f1462ad94de11/recovered.edits/15.seqid 2023-11-28 03:54:55,113 DEBUG [HFileArchiver-12] backup.HFileArchiver(596): Deleted hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/.tmp/data/default/testGetAfterCompaction/493f75b50f64227c277f1462ad94de11 2023-11-28 03:54:55,113 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(328): Archived testGetAfterCompaction regions 2023-11-28 03:54:55,113 DEBUG [PEWorker-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0 2023-11-28 03:54:55,114 DEBUG [PEWorker-4] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1] 2023-11-28 03:54:55,118 DEBUG [PEWorker-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e202311282ef74dc9675f4aaa91ba34d722ad5d26 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e202311282ef74dc9675f4aaa91ba34d722ad5d26 2023-11-28 03:54:55,118 DEBUG [PEWorker-4] backup.HFileArchiver(596): Deleted hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0 2023-11-28 03:54:55,121 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=78, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testGetAfterCompaction 2023-11-28 03:54:55,148 WARN [PEWorker-4] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of testGetAfterCompaction from hbase:meta 2023-11-28 03:54:55,150 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(421): Removing 'testGetAfterCompaction' descriptor. 2023-11-28 03:54:55,151 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=78, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testGetAfterCompaction 2023-11-28 03:54:55,151 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(411): Removing 'testGetAfterCompaction' from region states. 2023-11-28 03:54:55,151 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testGetAfterCompaction,,1701143688866.493f75b50f64227c277f1462ad94de11.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1701143695151"}]},"ts":"9223372036854775807"} 2023-11-28 03:54:55,152 INFO [PEWorker-4] hbase.MetaTableAccessor(1788): Deleted 1 regions from META 2023-11-28 03:54:55,152 DEBUG [PEWorker-4] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 493f75b50f64227c277f1462ad94de11, NAME => 'testGetAfterCompaction,,1701143688866.493f75b50f64227c277f1462ad94de11.', STARTKEY => '', ENDKEY => ''}] 2023-11-28 03:54:55,152 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(415): Marking 'testGetAfterCompaction' as deleted. 2023-11-28 03:54:55,153 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testGetAfterCompaction","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1701143695153"}]},"ts":"9223372036854775807"} 2023-11-28 03:54:55,154 INFO [PEWorker-4] hbase.MetaTableAccessor(1658): Deleted table testGetAfterCompaction state from META 2023-11-28 03:54:55,156 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(130): Finished pid=78, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testGetAfterCompaction 2023-11-28 03:54:55,157 INFO [PEWorker-4] procedure2.ProcedureExecutor(1409): Finished pid=78, state=SUCCESS; DeleteTableProcedure table=testGetAfterCompaction in 67 msec 2023-11-28 03:54:55,202 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(1230): Checking to see if procedure is done pid=78 2023-11-28 03:54:55,203 INFO [Listener at localhost/42853] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testGetAfterCompaction, procId: 78 completed 2023-11-28 03:54:55,215 INFO [Listener at localhost/42853] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testGetAfterCompaction Thread=240 (was 245), OpenFileDescriptor=627 (was 645), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=164 (was 170), ProcessCount=168 (was 170), AvailableMemoryMB=6849 (was 6912) 2023-11-28 03:54:55,225 INFO [Listener at localhost/42853] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testCompactionWithHFileLink Thread=240, OpenFileDescriptor=627, MaxFileDescriptor=60000, SystemLoadAverage=164, ProcessCount=168, AvailableMemoryMB=6849 2023-11-28 03:54:55,227 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testCompactionWithHFileLink', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} 2023-11-28 03:54:55,227 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] procedure2.ProcedureExecutor(1028): Stored pid=79, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testCompactionWithHFileLink 2023-11-28 03:54:55,229 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_PRE_OPERATION 2023-11-28 03:54:55,229 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(700): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testCompactionWithHFileLink" procId is: 79 2023-11-28 03:54:55,229 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-11-28 03:54:55,230 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(1230): Checking to see if procedure is done pid=79 2023-11-28 03:54:55,231 DEBUG [HFileArchiver-13] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/.tmp/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c 2023-11-28 03:54:55,231 DEBUG [HFileArchiver-14] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/.tmp/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5 2023-11-28 03:54:55,232 DEBUG [HFileArchiver-15] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/.tmp/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f 2023-11-28 03:54:55,232 DEBUG [HFileArchiver-14] backup.HFileArchiver(153): Directory hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/.tmp/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5 empty. 2023-11-28 03:54:55,232 DEBUG [HFileArchiver-15] backup.HFileArchiver(153): Directory hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/.tmp/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f empty. 2023-11-28 03:54:55,232 DEBUG [HFileArchiver-13] backup.HFileArchiver(153): Directory hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/.tmp/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c empty. 2023-11-28 03:54:55,233 DEBUG [HFileArchiver-14] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/.tmp/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5 2023-11-28 03:54:55,233 DEBUG [HFileArchiver-15] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/.tmp/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f 2023-11-28 03:54:55,233 DEBUG [HFileArchiver-13] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/.tmp/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c 2023-11-28 03:54:55,233 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(328): Archived testCompactionWithHFileLink regions 2023-11-28 03:54:55,330 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(1230): Checking to see if procedure is done pid=79 2023-11-28 03:54:55,531 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(1230): Checking to see if procedure is done pid=79 2023-11-28 03:54:55,650 DEBUG [PEWorker-5] util.FSTableDescriptors(570): Wrote into hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/.tmp/data/default/testCompactionWithHFileLink/.tabledesc/.tableinfo.0000000001 2023-11-28 03:54:55,651 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(7675): creating {ENCODED => 3d7f49de72f2e9e741f04714c6a7648c, NAME => 'testCompactionWithHFileLink,,1701143695226.3d7f49de72f2e9e741f04714c6a7648c.', STARTKEY => '', ENDKEY => '1'}, tableDescriptor='testCompactionWithHFileLink', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/.tmp 2023-11-28 03:54:55,652 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(7675): creating {ENCODED => 2aad05dc9c73dcbea66259e5c0a6958f, NAME => 'testCompactionWithHFileLink,2,1701143695226.2aad05dc9c73dcbea66259e5c0a6958f.', STARTKEY => '2', ENDKEY => ''}, tableDescriptor='testCompactionWithHFileLink', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/.tmp 2023-11-28 03:54:55,652 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(7675): creating {ENCODED => b3c2fdb68f55a3cbe8d29ee7b6ddd5e5, NAME => 'testCompactionWithHFileLink,1,1701143695226.b3c2fdb68f55a3cbe8d29ee7b6ddd5e5.', STARTKEY => '1', ENDKEY => '2'}, tableDescriptor='testCompactionWithHFileLink', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/.tmp 2023-11-28 03:54:55,666 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,2,1701143695226.2aad05dc9c73dcbea66259e5c0a6958f.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-28 03:54:55,666 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1604): Closing 2aad05dc9c73dcbea66259e5c0a6958f, disabling compactions & flushes 2023-11-28 03:54:55,666 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,2,1701143695226.2aad05dc9c73dcbea66259e5c0a6958f. 2023-11-28 03:54:55,666 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,2,1701143695226.2aad05dc9c73dcbea66259e5c0a6958f. 2023-11-28 03:54:55,666 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,2,1701143695226.2aad05dc9c73dcbea66259e5c0a6958f. after waiting 0 ms 2023-11-28 03:54:55,666 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,2,1701143695226.2aad05dc9c73dcbea66259e5c0a6958f. 2023-11-28 03:54:55,666 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,2,1701143695226.2aad05dc9c73dcbea66259e5c0a6958f. 2023-11-28 03:54:55,666 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1558): Region close journal for 2aad05dc9c73dcbea66259e5c0a6958f: 2023-11-28 03:54:55,833 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(1230): Checking to see if procedure is done pid=79 2023-11-28 03:54:56,059 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,1,1701143695226.b3c2fdb68f55a3cbe8d29ee7b6ddd5e5.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-28 03:54:56,059 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1604): Closing b3c2fdb68f55a3cbe8d29ee7b6ddd5e5, disabling compactions & flushes 2023-11-28 03:54:56,059 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,1,1701143695226.b3c2fdb68f55a3cbe8d29ee7b6ddd5e5. 2023-11-28 03:54:56,059 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,1,1701143695226.b3c2fdb68f55a3cbe8d29ee7b6ddd5e5. 2023-11-28 03:54:56,059 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,1,1701143695226.b3c2fdb68f55a3cbe8d29ee7b6ddd5e5. after waiting 0 ms 2023-11-28 03:54:56,059 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,1,1701143695226.b3c2fdb68f55a3cbe8d29ee7b6ddd5e5. 2023-11-28 03:54:56,059 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,1,1701143695226.b3c2fdb68f55a3cbe8d29ee7b6ddd5e5. 2023-11-28 03:54:56,059 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1558): Region close journal for b3c2fdb68f55a3cbe8d29ee7b6ddd5e5: 2023-11-28 03:54:56,063 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,,1701143695226.3d7f49de72f2e9e741f04714c6a7648c.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-28 03:54:56,063 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1604): Closing 3d7f49de72f2e9e741f04714c6a7648c, disabling compactions & flushes 2023-11-28 03:54:56,063 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,,1701143695226.3d7f49de72f2e9e741f04714c6a7648c. 2023-11-28 03:54:56,063 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,,1701143695226.3d7f49de72f2e9e741f04714c6a7648c. 2023-11-28 03:54:56,063 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,,1701143695226.3d7f49de72f2e9e741f04714c6a7648c. after waiting 0 ms 2023-11-28 03:54:56,063 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,,1701143695226.3d7f49de72f2e9e741f04714c6a7648c. 2023-11-28 03:54:56,063 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,,1701143695226.3d7f49de72f2e9e741f04714c6a7648c. 2023-11-28 03:54:56,063 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1558): Region close journal for 3d7f49de72f2e9e741f04714c6a7648c: 2023-11-28 03:54:56,065 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_ADD_TO_META 2023-11-28 03:54:56,066 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,2,1701143695226.2aad05dc9c73dcbea66259e5c0a6958f.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1701143696066"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1701143696066"}]},"ts":"1701143696066"} 2023-11-28 03:54:56,066 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,1,1701143695226.b3c2fdb68f55a3cbe8d29ee7b6ddd5e5.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1701143696066"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1701143696066"}]},"ts":"1701143696066"} 2023-11-28 03:54:56,066 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,,1701143695226.3d7f49de72f2e9e741f04714c6a7648c.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1701143696066"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1701143696066"}]},"ts":"1701143696066"} 2023-11-28 03:54:56,068 INFO [PEWorker-5] hbase.MetaTableAccessor(1496): Added 3 regions to meta. 2023-11-28 03:54:56,069 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-11-28 03:54:56,069 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1701143696069"}]},"ts":"1701143696069"} 2023-11-28 03:54:56,070 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=ENABLING in hbase:meta 2023-11-28 03:54:56,075 INFO [PEWorker-5] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=3d7f49de72f2e9e741f04714c6a7648c, ASSIGN}, {pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=b3c2fdb68f55a3cbe8d29ee7b6ddd5e5, ASSIGN}, {pid=82, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=2aad05dc9c73dcbea66259e5c0a6958f, ASSIGN}] 2023-11-28 03:54:56,076 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=3d7f49de72f2e9e741f04714c6a7648c, ASSIGN 2023-11-28 03:54:56,076 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=b3c2fdb68f55a3cbe8d29ee7b6ddd5e5, ASSIGN 2023-11-28 03:54:56,076 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=82, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=2aad05dc9c73dcbea66259e5c0a6958f, ASSIGN 2023-11-28 03:54:56,077 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=b3c2fdb68f55a3cbe8d29ee7b6ddd5e5, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,41203,1701143600653; forceNewPlan=false, retain=false 2023-11-28 03:54:56,077 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=3d7f49de72f2e9e741f04714c6a7648c, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,41203,1701143600653; forceNewPlan=false, retain=false 2023-11-28 03:54:56,077 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=82, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=2aad05dc9c73dcbea66259e5c0a6958f, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,41203,1701143600653; forceNewPlan=false, retain=false 2023-11-28 03:54:56,228 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=80 updating hbase:meta row=3d7f49de72f2e9e741f04714c6a7648c, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:56,228 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=81 updating hbase:meta row=b3c2fdb68f55a3cbe8d29ee7b6ddd5e5, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:56,228 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=82 updating hbase:meta row=2aad05dc9c73dcbea66259e5c0a6958f, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:56,229 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,,1701143695226.3d7f49de72f2e9e741f04714c6a7648c.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1701143696228"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1701143696228"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1701143696228"}]},"ts":"1701143696228"} 2023-11-28 03:54:56,229 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,1,1701143695226.b3c2fdb68f55a3cbe8d29ee7b6ddd5e5.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1701143696228"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1701143696228"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1701143696228"}]},"ts":"1701143696228"} 2023-11-28 03:54:56,229 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,2,1701143695226.2aad05dc9c73dcbea66259e5c0a6958f.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1701143696228"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1701143696228"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1701143696228"}]},"ts":"1701143696228"} 2023-11-28 03:54:56,230 INFO [PEWorker-3] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=83, ppid=80, state=RUNNABLE; OpenRegionProcedure 3d7f49de72f2e9e741f04714c6a7648c, server=jenkins-hbase4.apache.org,41203,1701143600653}] 2023-11-28 03:54:56,231 INFO [PEWorker-4] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=84, ppid=81, state=RUNNABLE; OpenRegionProcedure b3c2fdb68f55a3cbe8d29ee7b6ddd5e5, server=jenkins-hbase4.apache.org,41203,1701143600653}] 2023-11-28 03:54:56,233 INFO [PEWorker-5] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=85, ppid=82, state=RUNNABLE; OpenRegionProcedure 2aad05dc9c73dcbea66259e5c0a6958f, server=jenkins-hbase4.apache.org,41203,1701143600653}] 2023-11-28 03:54:56,333 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(1230): Checking to see if procedure is done pid=79 2023-11-28 03:54:56,384 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:56,387 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testCompactionWithHFileLink,,1701143695226.3d7f49de72f2e9e741f04714c6a7648c. 2023-11-28 03:54:56,387 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 3d7f49de72f2e9e741f04714c6a7648c, NAME => 'testCompactionWithHFileLink,,1701143695226.3d7f49de72f2e9e741f04714c6a7648c.', STARTKEY => '', ENDKEY => '1'} 2023-11-28 03:54:56,387 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink 3d7f49de72f2e9e741f04714c6a7648c 2023-11-28 03:54:56,387 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,,1701143695226.3d7f49de72f2e9e741f04714c6a7648c.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-28 03:54:56,387 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 3d7f49de72f2e9e741f04714c6a7648c 2023-11-28 03:54:56,387 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 3d7f49de72f2e9e741f04714c6a7648c 2023-11-28 03:54:56,388 INFO [StoreOpener-3d7f49de72f2e9e741f04714c6a7648c-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 3d7f49de72f2e9e741f04714c6a7648c 2023-11-28 03:54:56,390 DEBUG [StoreOpener-3d7f49de72f2e9e741f04714c6a7648c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family1 2023-11-28 03:54:56,390 DEBUG [StoreOpener-3d7f49de72f2e9e741f04714c6a7648c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family1 2023-11-28 03:54:56,390 INFO [StoreOpener-3d7f49de72f2e9e741f04714c6a7648c-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 3d7f49de72f2e9e741f04714c6a7648c columnFamilyName family1 2023-11-28 03:54:56,391 INFO [StoreOpener-3d7f49de72f2e9e741f04714c6a7648c-1] regionserver.HStore(310): Store=3d7f49de72f2e9e741f04714c6a7648c/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-28 03:54:56,392 INFO [StoreOpener-3d7f49de72f2e9e741f04714c6a7648c-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 3d7f49de72f2e9e741f04714c6a7648c 2023-11-28 03:54:56,393 DEBUG [StoreOpener-3d7f49de72f2e9e741f04714c6a7648c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family2 2023-11-28 03:54:56,393 DEBUG [StoreOpener-3d7f49de72f2e9e741f04714c6a7648c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family2 2023-11-28 03:54:56,393 INFO [StoreOpener-3d7f49de72f2e9e741f04714c6a7648c-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 3d7f49de72f2e9e741f04714c6a7648c columnFamilyName family2 2023-11-28 03:54:56,394 INFO [StoreOpener-3d7f49de72f2e9e741f04714c6a7648c-1] regionserver.HStore(310): Store=3d7f49de72f2e9e741f04714c6a7648c/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-28 03:54:56,395 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c 2023-11-28 03:54:56,395 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c 2023-11-28 03:54:56,398 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-11-28 03:54:56,399 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 3d7f49de72f2e9e741f04714c6a7648c 2023-11-28 03:54:56,401 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-11-28 03:54:56,402 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 3d7f49de72f2e9e741f04714c6a7648c; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67036004, jitterRate=-0.0010856986045837402}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-11-28 03:54:56,402 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 3d7f49de72f2e9e741f04714c6a7648c: 2023-11-28 03:54:56,403 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2339): Post open deploy tasks for testCompactionWithHFileLink,,1701143695226.3d7f49de72f2e9e741f04714c6a7648c., pid=83, masterSystemTime=1701143696384 2023-11-28 03:54:56,404 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2366): Finished post open deploy task for testCompactionWithHFileLink,,1701143695226.3d7f49de72f2e9e741f04714c6a7648c. 2023-11-28 03:54:56,404 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testCompactionWithHFileLink,,1701143695226.3d7f49de72f2e9e741f04714c6a7648c. 2023-11-28 03:54:56,404 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testCompactionWithHFileLink,1,1701143695226.b3c2fdb68f55a3cbe8d29ee7b6ddd5e5. 2023-11-28 03:54:56,404 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => b3c2fdb68f55a3cbe8d29ee7b6ddd5e5, NAME => 'testCompactionWithHFileLink,1,1701143695226.b3c2fdb68f55a3cbe8d29ee7b6ddd5e5.', STARTKEY => '1', ENDKEY => '2'} 2023-11-28 03:54:56,404 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=80 updating hbase:meta row=3d7f49de72f2e9e741f04714c6a7648c, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:56,405 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink b3c2fdb68f55a3cbe8d29ee7b6ddd5e5 2023-11-28 03:54:56,405 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,,1701143695226.3d7f49de72f2e9e741f04714c6a7648c.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1701143696404"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1701143696404"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1701143696404"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1701143696404"}]},"ts":"1701143696404"} 2023-11-28 03:54:56,405 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,1,1701143695226.b3c2fdb68f55a3cbe8d29ee7b6ddd5e5.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-28 03:54:56,405 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for b3c2fdb68f55a3cbe8d29ee7b6ddd5e5 2023-11-28 03:54:56,405 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for b3c2fdb68f55a3cbe8d29ee7b6ddd5e5 2023-11-28 03:54:56,406 INFO [StoreOpener-b3c2fdb68f55a3cbe8d29ee7b6ddd5e5-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region b3c2fdb68f55a3cbe8d29ee7b6ddd5e5 2023-11-28 03:54:56,407 DEBUG [StoreOpener-b3c2fdb68f55a3cbe8d29ee7b6ddd5e5-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family1 2023-11-28 03:54:56,407 INFO [PEWorker-3] procedure2.ProcedureExecutor(1825): Finished subprocedure pid=83, resume processing ppid=80 2023-11-28 03:54:56,407 DEBUG [StoreOpener-b3c2fdb68f55a3cbe8d29ee7b6ddd5e5-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family1 2023-11-28 03:54:56,408 INFO [StoreOpener-b3c2fdb68f55a3cbe8d29ee7b6ddd5e5-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region b3c2fdb68f55a3cbe8d29ee7b6ddd5e5 columnFamilyName family1 2023-11-28 03:54:56,408 INFO [PEWorker-3] procedure2.ProcedureExecutor(1409): Finished pid=83, ppid=80, state=SUCCESS; OpenRegionProcedure 3d7f49de72f2e9e741f04714c6a7648c, server=jenkins-hbase4.apache.org,41203,1701143600653 in 176 msec 2023-11-28 03:54:56,408 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=80, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=3d7f49de72f2e9e741f04714c6a7648c, ASSIGN in 333 msec 2023-11-28 03:54:56,409 INFO [StoreOpener-b3c2fdb68f55a3cbe8d29ee7b6ddd5e5-1] regionserver.HStore(310): Store=b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-28 03:54:56,409 INFO [StoreOpener-b3c2fdb68f55a3cbe8d29ee7b6ddd5e5-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region b3c2fdb68f55a3cbe8d29ee7b6ddd5e5 2023-11-28 03:54:56,410 DEBUG [StoreOpener-b3c2fdb68f55a3cbe8d29ee7b6ddd5e5-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family2 2023-11-28 03:54:56,410 DEBUG [StoreOpener-b3c2fdb68f55a3cbe8d29ee7b6ddd5e5-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family2 2023-11-28 03:54:56,410 INFO [StoreOpener-b3c2fdb68f55a3cbe8d29ee7b6ddd5e5-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region b3c2fdb68f55a3cbe8d29ee7b6ddd5e5 columnFamilyName family2 2023-11-28 03:54:56,411 INFO [StoreOpener-b3c2fdb68f55a3cbe8d29ee7b6ddd5e5-1] regionserver.HStore(310): Store=b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-28 03:54:56,412 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5 2023-11-28 03:54:56,412 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5 2023-11-28 03:54:56,414 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-11-28 03:54:56,414 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for b3c2fdb68f55a3cbe8d29ee7b6ddd5e5 2023-11-28 03:54:56,416 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-11-28 03:54:56,417 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened b3c2fdb68f55a3cbe8d29ee7b6ddd5e5; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61922756, jitterRate=-0.07727903127670288}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-11-28 03:54:56,417 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for b3c2fdb68f55a3cbe8d29ee7b6ddd5e5: 2023-11-28 03:54:56,417 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2339): Post open deploy tasks for testCompactionWithHFileLink,1,1701143695226.b3c2fdb68f55a3cbe8d29ee7b6ddd5e5., pid=84, masterSystemTime=1701143696384 2023-11-28 03:54:56,418 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2366): Finished post open deploy task for testCompactionWithHFileLink,1,1701143695226.b3c2fdb68f55a3cbe8d29ee7b6ddd5e5. 2023-11-28 03:54:56,418 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testCompactionWithHFileLink,1,1701143695226.b3c2fdb68f55a3cbe8d29ee7b6ddd5e5. 2023-11-28 03:54:56,418 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testCompactionWithHFileLink,2,1701143695226.2aad05dc9c73dcbea66259e5c0a6958f. 2023-11-28 03:54:56,419 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 2aad05dc9c73dcbea66259e5c0a6958f, NAME => 'testCompactionWithHFileLink,2,1701143695226.2aad05dc9c73dcbea66259e5c0a6958f.', STARTKEY => '2', ENDKEY => ''} 2023-11-28 03:54:56,419 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=81 updating hbase:meta row=b3c2fdb68f55a3cbe8d29ee7b6ddd5e5, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:56,419 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,1,1701143695226.b3c2fdb68f55a3cbe8d29ee7b6ddd5e5.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1701143696419"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1701143696419"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1701143696419"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1701143696419"}]},"ts":"1701143696419"} 2023-11-28 03:54:56,419 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink 2aad05dc9c73dcbea66259e5c0a6958f 2023-11-28 03:54:56,419 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,2,1701143695226.2aad05dc9c73dcbea66259e5c0a6958f.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-28 03:54:56,419 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 2aad05dc9c73dcbea66259e5c0a6958f 2023-11-28 03:54:56,419 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 2aad05dc9c73dcbea66259e5c0a6958f 2023-11-28 03:54:56,420 INFO [StoreOpener-2aad05dc9c73dcbea66259e5c0a6958f-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 2aad05dc9c73dcbea66259e5c0a6958f 2023-11-28 03:54:56,421 DEBUG [StoreOpener-2aad05dc9c73dcbea66259e5c0a6958f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family1 2023-11-28 03:54:56,421 DEBUG [StoreOpener-2aad05dc9c73dcbea66259e5c0a6958f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family1 2023-11-28 03:54:56,421 INFO [PEWorker-2] procedure2.ProcedureExecutor(1825): Finished subprocedure pid=84, resume processing ppid=81 2023-11-28 03:54:56,422 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=84, ppid=81, state=SUCCESS; OpenRegionProcedure b3c2fdb68f55a3cbe8d29ee7b6ddd5e5, server=jenkins-hbase4.apache.org,41203,1701143600653 in 189 msec 2023-11-28 03:54:56,422 INFO [StoreOpener-2aad05dc9c73dcbea66259e5c0a6958f-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 2aad05dc9c73dcbea66259e5c0a6958f columnFamilyName family1 2023-11-28 03:54:56,422 INFO [StoreOpener-2aad05dc9c73dcbea66259e5c0a6958f-1] regionserver.HStore(310): Store=2aad05dc9c73dcbea66259e5c0a6958f/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-28 03:54:56,422 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=81, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=b3c2fdb68f55a3cbe8d29ee7b6ddd5e5, ASSIGN in 348 msec 2023-11-28 03:54:56,423 INFO [StoreOpener-2aad05dc9c73dcbea66259e5c0a6958f-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 2aad05dc9c73dcbea66259e5c0a6958f 2023-11-28 03:54:56,424 DEBUG [StoreOpener-2aad05dc9c73dcbea66259e5c0a6958f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family2 2023-11-28 03:54:56,424 DEBUG [StoreOpener-2aad05dc9c73dcbea66259e5c0a6958f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family2 2023-11-28 03:54:56,424 INFO [StoreOpener-2aad05dc9c73dcbea66259e5c0a6958f-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 2aad05dc9c73dcbea66259e5c0a6958f columnFamilyName family2 2023-11-28 03:54:56,425 INFO [StoreOpener-2aad05dc9c73dcbea66259e5c0a6958f-1] regionserver.HStore(310): Store=2aad05dc9c73dcbea66259e5c0a6958f/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-28 03:54:56,425 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f 2023-11-28 03:54:56,426 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f 2023-11-28 03:54:56,427 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-11-28 03:54:56,428 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 2aad05dc9c73dcbea66259e5c0a6958f 2023-11-28 03:54:56,429 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-11-28 03:54:56,430 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 2aad05dc9c73dcbea66259e5c0a6958f; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63329548, jitterRate=-0.05631619691848755}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-11-28 03:54:56,430 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 2aad05dc9c73dcbea66259e5c0a6958f: 2023-11-28 03:54:56,430 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2339): Post open deploy tasks for testCompactionWithHFileLink,2,1701143695226.2aad05dc9c73dcbea66259e5c0a6958f., pid=85, masterSystemTime=1701143696384 2023-11-28 03:54:56,431 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2366): Finished post open deploy task for testCompactionWithHFileLink,2,1701143695226.2aad05dc9c73dcbea66259e5c0a6958f. 2023-11-28 03:54:56,432 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testCompactionWithHFileLink,2,1701143695226.2aad05dc9c73dcbea66259e5c0a6958f. 2023-11-28 03:54:56,432 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=82 updating hbase:meta row=2aad05dc9c73dcbea66259e5c0a6958f, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:56,432 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,2,1701143695226.2aad05dc9c73dcbea66259e5c0a6958f.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1701143696432"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1701143696432"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1701143696432"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1701143696432"}]},"ts":"1701143696432"} 2023-11-28 03:54:56,434 INFO [PEWorker-4] procedure2.ProcedureExecutor(1825): Finished subprocedure pid=85, resume processing ppid=82 2023-11-28 03:54:56,434 INFO [PEWorker-4] procedure2.ProcedureExecutor(1409): Finished pid=85, ppid=82, state=SUCCESS; OpenRegionProcedure 2aad05dc9c73dcbea66259e5c0a6958f, server=jenkins-hbase4.apache.org,41203,1701143600653 in 201 msec 2023-11-28 03:54:56,436 INFO [PEWorker-3] procedure2.ProcedureExecutor(1825): Finished subprocedure pid=82, resume processing ppid=79 2023-11-28 03:54:56,436 INFO [PEWorker-3] procedure2.ProcedureExecutor(1409): Finished pid=82, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=2aad05dc9c73dcbea66259e5c0a6958f, ASSIGN in 359 msec 2023-11-28 03:54:56,436 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-11-28 03:54:56,436 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1701143696436"}]},"ts":"1701143696436"} 2023-11-28 03:54:56,437 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=ENABLED in hbase:meta 2023-11-28 03:54:56,440 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_POST_OPERATION 2023-11-28 03:54:56,441 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=79, state=SUCCESS; CreateTableProcedure table=testCompactionWithHFileLink in 1.2130 sec 2023-11-28 03:54:57,335 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(1230): Checking to see if procedure is done pid=79 2023-11-28 03:54:57,335 INFO [Listener at localhost/42853] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testCompactionWithHFileLink, procId: 79 completed 2023-11-28 03:54:57,342 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,,1701143695226.3d7f49de72f2e9e741f04714c6a7648c. with WAL disabled. Data may be lost in the event of a crash. 2023-11-28 03:54:57,344 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,1,1701143695226.b3c2fdb68f55a3cbe8d29ee7b6ddd5e5. with WAL disabled. Data may be lost in the event of a crash. 2023-11-28 03:54:57,344 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,2,1701143695226.2aad05dc9c73dcbea66259e5c0a6958f. with WAL disabled. Data may be lost in the event of a crash. 2023-11-28 03:54:57,347 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-28 03:54:57,349 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] procedure.ProcedureCoordinator(165): Submitting procedure testCompactionWithHFileLink 2023-11-28 03:54:57,349 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testCompactionWithHFileLink' 2023-11-28 03:54:57,349 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:54:57,349 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testCompactionWithHFileLink' starting 'acquire' 2023-11-28 03:54:57,349 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testCompactionWithHFileLink', kicking off acquire phase on members. 2023-11-28 03:54:57,349 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-28 03:54:57,349 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-11-28 03:54:57,351 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:57,351 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:54:57,351 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:54:57,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:54:57,351 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:57,351 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-28 03:54:57,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-11-28 03:54:57,352 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-28 03:54:57,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-28 03:54:57,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-11-28 03:54:57,352 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testCompactionWithHFileLink 2023-11-28 03:54:57,352 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testCompactionWithHFileLink 2023-11-28 03:54:57,352 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testCompactionWithHFileLink' with timeout 60000ms 2023-11-28 03:54:57,352 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:54:57,352 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testCompactionWithHFileLink' starting 'acquire' stage 2023-11-28 03:54:57,352 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,,1701143695226.3d7f49de72f2e9e741f04714c6a7648c. 2023-11-28 03:54:57,352 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,,1701143695226.3d7f49de72f2e9e741f04714c6a7648c. started... 2023-11-28 03:54:57,358 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 3d7f49de72f2e9e741f04714c6a7648c 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-11-28 03:54:57,358 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-11-28 03:54:57,358 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-28 03:54:57,358 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,1,1701143695226.b3c2fdb68f55a3cbe8d29ee7b6ddd5e5. 2023-11-28 03:54:57,358 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,1,1701143695226.b3c2fdb68f55a3cbe8d29ee7b6ddd5e5. started... 2023-11-28 03:54:57,358 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,2,1701143695226.2aad05dc9c73dcbea66259e5c0a6958f. 2023-11-28 03:54:57,358 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,2,1701143695226.2aad05dc9c73dcbea66259e5c0a6958f. started... 2023-11-28 03:54:57,358 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing b3c2fdb68f55a3cbe8d29ee7b6ddd5e5 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-11-28 03:54:57,359 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 2aad05dc9c73dcbea66259e5c0a6958f 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-11-28 03:54:57,777 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20231128ae21d695d78b47819031a63d23fd1adf to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20231128ae21d695d78b47819031a63d23fd1adf 2023-11-28 03:54:57,778 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20231128dec752e8fb644a0a80d6b4fd043402c1 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20231128dec752e8fb644a0a80d6b4fd043402c1 2023-11-28 03:54:57,778 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023112838c9c1bb8cde4e7f815926db3a63a195 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023112838c9c1bb8cde4e7f815926db3a63a195 2023-11-28 03:54:57,786 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/.tmp/family1/9cba99270b974d6b9181045f6f7ee6bc 2023-11-28 03:54:57,787 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/.tmp/family1/88a76a3cbca146b6bb71214ecc4a9c87 2023-11-28 03:54:58,151 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testCompactionWithHFileLink' 2023-11-28 03:54:58,183 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/.tmp/family1/5bd4594355ca45ff98e17e8e298757d3 2023-11-28 03:54:58,202 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202311284073e83e8d4242548321ae15f6b216d4 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202311284073e83e8d4242548321ae15f6b216d4 2023-11-28 03:54:58,202 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202311284dec3a2023324667bf8796b80a2f3bce to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202311284dec3a2023324667bf8796b80a2f3bce 2023-11-28 03:54:58,206 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/.tmp/family2/ee2d50b684b2479e8a02236cf5083faa 2023-11-28 03:54:58,210 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/.tmp/family1/88a76a3cbca146b6bb71214ecc4a9c87 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family1/88a76a3cbca146b6bb71214ecc4a9c87 2023-11-28 03:54:58,214 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family1/88a76a3cbca146b6bb71214ecc4a9c87, entries=4, sequenceid=5, filesize=5.4 K 2023-11-28 03:54:58,215 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/.tmp/family2/ee2d50b684b2479e8a02236cf5083faa as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family2/ee2d50b684b2479e8a02236cf5083faa 2023-11-28 03:54:58,218 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family2/ee2d50b684b2479e8a02236cf5083faa, entries=2, sequenceid=5, filesize=5.1 K 2023-11-28 03:54:58,219 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for b3c2fdb68f55a3cbe8d29ee7b6ddd5e5 in 860ms, sequenceid=5, compaction requested=false 2023-11-28 03:54:58,219 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for b3c2fdb68f55a3cbe8d29ee7b6ddd5e5: 2023-11-28 03:54:58,219 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,1,1701143695226.b3c2fdb68f55a3cbe8d29ee7b6ddd5e5. 2023-11-28 03:54:58,219 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-11-28 03:54:58,597 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202311284eac4cd4063e44bf9de8ec54912c3ff0 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202311284eac4cd4063e44bf9de8ec54912c3ff0 2023-11-28 03:54:58,602 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/.tmp/family2/a79a53b7062d4e53b9199b02c23f6c24 2023-11-28 03:54:58,605 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/.tmp/family1/5bd4594355ca45ff98e17e8e298757d3 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family1/5bd4594355ca45ff98e17e8e298757d3 2023-11-28 03:54:58,607 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/.tmp/family2/cfbbdda0c47f440ab7343ce95b4baaf5 2023-11-28 03:54:58,609 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family1/5bd4594355ca45ff98e17e8e298757d3, entries=4, sequenceid=5, filesize=5.4 K 2023-11-28 03:54:58,609 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/.tmp/family2/a79a53b7062d4e53b9199b02c23f6c24 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family2/a79a53b7062d4e53b9199b02c23f6c24 2023-11-28 03:54:58,611 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/.tmp/family1/9cba99270b974d6b9181045f6f7ee6bc as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family1/9cba99270b974d6b9181045f6f7ee6bc 2023-11-28 03:54:58,614 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family2/a79a53b7062d4e53b9199b02c23f6c24, entries=2, sequenceid=5, filesize=5.1 K 2023-11-28 03:54:58,614 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 2aad05dc9c73dcbea66259e5c0a6958f in 1255ms, sequenceid=5, compaction requested=false 2023-11-28 03:54:58,615 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 2aad05dc9c73dcbea66259e5c0a6958f: 2023-11-28 03:54:58,615 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,2,1701143695226.2aad05dc9c73dcbea66259e5c0a6958f. 2023-11-28 03:54:58,615 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-11-28 03:54:58,615 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family1/9cba99270b974d6b9181045f6f7ee6bc, entries=4, sequenceid=5, filesize=5.4 K 2023-11-28 03:54:58,615 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/.tmp/family2/cfbbdda0c47f440ab7343ce95b4baaf5 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family2/cfbbdda0c47f440ab7343ce95b4baaf5 2023-11-28 03:54:58,619 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family2/cfbbdda0c47f440ab7343ce95b4baaf5, entries=2, sequenceid=5, filesize=5.1 K 2023-11-28 03:54:58,619 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 3d7f49de72f2e9e741f04714c6a7648c in 1261ms, sequenceid=5, compaction requested=false 2023-11-28 03:54:58,619 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 3d7f49de72f2e9e741f04714c6a7648c: 2023-11-28 03:54:58,620 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,,1701143695226.3d7f49de72f2e9e741f04714c6a7648c. 2023-11-28 03:54:58,620 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-11-28 03:54:58,620 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-11-28 03:54:58,620 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:58,620 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testCompactionWithHFileLink' locally acquired 2023-11-28 03:54:58,620 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure (testCompactionWithHFileLink) in zk 2023-11-28 03:54:58,623 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:58,623 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-28 03:54:58,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:58,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:54:58,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:54:58,623 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-28 03:54:58,623 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testCompactionWithHFileLink' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-28 03:54:58,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:54:58,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:54:58,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-11-28 03:54:58,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:58,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:54:58,624 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure 'testCompactionWithHFileLink' on coordinator 2023-11-28 03:54:58,624 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@30411fd9[Count = 0] remaining members to acquire global barrier 2023-11-28 03:54:58,624 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testCompactionWithHFileLink' starting 'in-barrier' execution. 2023-11-28 03:54:58,625 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-28 03:54:58,626 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-28 03:54:58,626 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-28 03:54:58,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-28 03:54:58,626 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testCompactionWithHFileLink' received 'reached' from coordinator. 2023-11-28 03:54:58,626 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testCompactionWithHFileLink' locally completed 2023-11-28 03:54:58,626 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testCompactionWithHFileLink' completed for member 'jenkins-hbase4.apache.org,41203,1701143600653' in zk 2023-11-28 03:54:58,626 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:58,626 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-28 03:54:58,627 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testCompactionWithHFileLink' has notified controller of completion 2023-11-28 03:54:58,627 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:58,627 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:54:58,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:58,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:54:58,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:54:58,627 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testCompactionWithHFileLink' completed. 2023-11-28 03:54:58,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:54:58,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:54:58,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-11-28 03:54:58,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:58,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:54:58,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-11-28 03:54:58,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:58,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testCompactionWithHFileLink' member 'jenkins-hbase4.apache.org,41203,1701143600653': 2023-11-28 03:54:58,629 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' released barrier for procedure'testCompactionWithHFileLink', counting down latch. Waiting for 0 more 2023-11-28 03:54:58,629 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testCompactionWithHFileLink' execution completed 2023-11-28 03:54:58,629 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-28 03:54:58,629 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-28 03:54:58,630 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testCompactionWithHFileLink 2023-11-28 03:54:58,630 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testCompactionWithHFileLinkincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-28 03:54:58,631 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-28 03:54:58,631 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-28 03:54:58,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-28 03:54:58,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:54:58,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:54:58,631 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-28 03:54:58,631 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:54:58,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-28 03:54:58,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:54:58,632 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:58,632 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:54:58,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-11-28 03:54:58,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:54:58,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:54:58,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-28 03:54:58,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-11-28 03:54:58,633 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:58,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:58,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:54:58,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-11-28 03:54:58,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:58,635 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:58,635 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:54:58,635 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-11-28 03:54:58,635 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:54:58,635 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:54:58,635 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-11-28 03:54:58,635 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:54:58,636 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:58,636 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-28 03:54:58,636 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-28 03:54:58,636 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-28 03:54:58,635 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testCompactionWithHFileLink' 2023-11-28 03:54:58,636 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-28 03:54:58,636 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(399): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink because node does not exist (not an error) 2023-11-28 03:54:58,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:54:58,636 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testCompactionWithHFileLink'' to complete. (max 300000 ms per retry) 2023-11-28 03:54:58,636 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-28 03:54:58,636 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:54:58,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:54:58,736 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-28 03:54:58,737 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-28 03:54:58,738 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,,1701143695226.3d7f49de72f2e9e741f04714c6a7648c. with WAL disabled. Data may be lost in the event of a crash. 2023-11-28 03:54:58,739 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,1,1701143695226.b3c2fdb68f55a3cbe8d29ee7b6ddd5e5. with WAL disabled. Data may be lost in the event of a crash. 2023-11-28 03:54:58,739 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,2,1701143695226.2aad05dc9c73dcbea66259e5c0a6958f. with WAL disabled. Data may be lost in the event of a crash. 2023-11-28 03:54:58,741 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-28 03:54:58,743 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] procedure.ProcedureCoordinator(143): Procedure testCompactionWithHFileLink was in running list but was completed. Accepting new attempt. 2023-11-28 03:54:58,743 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] procedure.ProcedureCoordinator(165): Submitting procedure testCompactionWithHFileLink 2023-11-28 03:54:58,743 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testCompactionWithHFileLink' 2023-11-28 03:54:58,743 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:54:58,743 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testCompactionWithHFileLink' starting 'acquire' 2023-11-28 03:54:58,743 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testCompactionWithHFileLink', kicking off acquire phase on members. 2023-11-28 03:54:58,744 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-28 03:54:58,744 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-11-28 03:54:58,745 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:58,745 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:54:58,745 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:54:58,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:54:58,746 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:54:58,746 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-28 03:54:58,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-11-28 03:54:58,746 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-28 03:54:58,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-28 03:54:58,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-11-28 03:54:58,747 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testCompactionWithHFileLink 2023-11-28 03:54:58,748 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testCompactionWithHFileLink is still present, removing 2023-11-28 03:54:58,748 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testCompactionWithHFileLink 2023-11-28 03:54:58,748 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testCompactionWithHFileLink' with timeout 60000ms 2023-11-28 03:54:58,748 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:54:58,748 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testCompactionWithHFileLink' starting 'acquire' stage 2023-11-28 03:54:58,753 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,,1701143695226.3d7f49de72f2e9e741f04714c6a7648c. 2023-11-28 03:54:58,753 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,,1701143695226.3d7f49de72f2e9e741f04714c6a7648c. started... 2023-11-28 03:54:58,753 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,1,1701143695226.b3c2fdb68f55a3cbe8d29ee7b6ddd5e5. 2023-11-28 03:54:58,753 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-11-28 03:54:58,753 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-28 03:54:58,753 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 3d7f49de72f2e9e741f04714c6a7648c 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-11-28 03:54:58,753 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,1,1701143695226.b3c2fdb68f55a3cbe8d29ee7b6ddd5e5. started... 2023-11-28 03:54:58,753 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,2,1701143695226.2aad05dc9c73dcbea66259e5c0a6958f. 2023-11-28 03:54:58,753 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,2,1701143695226.2aad05dc9c73dcbea66259e5c0a6958f. started... 2023-11-28 03:54:58,753 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing b3c2fdb68f55a3cbe8d29ee7b6ddd5e5 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-11-28 03:54:58,753 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 2aad05dc9c73dcbea66259e5c0a6958f 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-11-28 03:54:58,786 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202311287fb879a012e040c992a5c4c12e52bff2 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202311287fb879a012e040c992a5c4c12e52bff2 2023-11-28 03:54:58,794 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/.tmp/family1/299d98e37656434ea5b2482137c58710 2023-11-28 03:54:58,809 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202311283312fc03dd134b7bab571a71f58de4a3 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202311283312fc03dd134b7bab571a71f58de4a3 2023-11-28 03:54:59,180 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023112864ca880632ff4b81b856e56f97a51ba5 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e2023112864ca880632ff4b81b856e56f97a51ba5 2023-11-28 03:54:59,187 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20231128fb09572884984543b4a9faae21b5679c to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20231128fb09572884984543b4a9faae21b5679c 2023-11-28 03:54:59,214 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/.tmp/family2/75ff1f4a28e9466e86a2f50fe4b3ba7c 2023-11-28 03:54:59,218 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/.tmp/family1/299d98e37656434ea5b2482137c58710 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family1/299d98e37656434ea5b2482137c58710 2023-11-28 03:54:59,222 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family1/299d98e37656434ea5b2482137c58710, entries=4, sequenceid=9, filesize=5.4 K 2023-11-28 03:54:59,223 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/.tmp/family2/75ff1f4a28e9466e86a2f50fe4b3ba7c as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family2/75ff1f4a28e9466e86a2f50fe4b3ba7c 2023-11-28 03:54:59,227 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family2/75ff1f4a28e9466e86a2f50fe4b3ba7c, entries=2, sequenceid=9, filesize=5.1 K 2023-11-28 03:54:59,228 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for b3c2fdb68f55a3cbe8d29ee7b6ddd5e5 in 474ms, sequenceid=9, compaction requested=false 2023-11-28 03:54:59,228 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for b3c2fdb68f55a3cbe8d29ee7b6ddd5e5: 2023-11-28 03:54:59,228 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,1,1701143695226.b3c2fdb68f55a3cbe8d29ee7b6ddd5e5. 2023-11-28 03:54:59,228 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-11-28 03:54:59,586 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/.tmp/family1/149c75fc3d8a444bbd7d2180c2614583 2023-11-28 03:54:59,592 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/.tmp/family1/ef8ca9f746b4468d8da6b892ab104927 2023-11-28 03:55:00,001 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20231128310f916e533545709f94215002f0c70b to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20231128310f916e533545709f94215002f0c70b 2023-11-28 03:55:00,006 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202311284edd51e4fb9f46e894cf420c094d1251 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202311284edd51e4fb9f46e894cf420c094d1251 2023-11-28 03:55:00,406 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/.tmp/family2/14783dff7d074f04ac3290bc0e4a92ba 2023-11-28 03:55:00,411 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/.tmp/family2/a5d2b1b5f59d4745beda10e8ec8e729f 2023-11-28 03:55:00,411 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/.tmp/family1/149c75fc3d8a444bbd7d2180c2614583 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family1/149c75fc3d8a444bbd7d2180c2614583 2023-11-28 03:55:00,415 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/.tmp/family1/ef8ca9f746b4468d8da6b892ab104927 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family1/ef8ca9f746b4468d8da6b892ab104927 2023-11-28 03:55:00,416 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family1/149c75fc3d8a444bbd7d2180c2614583, entries=4, sequenceid=9, filesize=5.4 K 2023-11-28 03:55:00,417 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/.tmp/family2/14783dff7d074f04ac3290bc0e4a92ba as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family2/14783dff7d074f04ac3290bc0e4a92ba 2023-11-28 03:55:00,419 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family1/ef8ca9f746b4468d8da6b892ab104927, entries=4, sequenceid=9, filesize=5.4 K 2023-11-28 03:55:00,420 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/.tmp/family2/a5d2b1b5f59d4745beda10e8ec8e729f as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family2/a5d2b1b5f59d4745beda10e8ec8e729f 2023-11-28 03:55:00,421 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family2/14783dff7d074f04ac3290bc0e4a92ba, entries=2, sequenceid=9, filesize=5.1 K 2023-11-28 03:55:00,422 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 3d7f49de72f2e9e741f04714c6a7648c in 1669ms, sequenceid=9, compaction requested=false 2023-11-28 03:55:00,422 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 3d7f49de72f2e9e741f04714c6a7648c: 2023-11-28 03:55:00,422 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,,1701143695226.3d7f49de72f2e9e741f04714c6a7648c. 2023-11-28 03:55:00,422 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-11-28 03:55:00,424 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family2/a5d2b1b5f59d4745beda10e8ec8e729f, entries=2, sequenceid=9, filesize=5.1 K 2023-11-28 03:55:00,424 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 2aad05dc9c73dcbea66259e5c0a6958f in 1671ms, sequenceid=9, compaction requested=false 2023-11-28 03:55:00,425 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 2aad05dc9c73dcbea66259e5c0a6958f: 2023-11-28 03:55:00,425 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,2,1701143695226.2aad05dc9c73dcbea66259e5c0a6958f. 2023-11-28 03:55:00,425 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-11-28 03:55:00,425 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-11-28 03:55:00,425 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:00,425 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testCompactionWithHFileLink' locally acquired 2023-11-28 03:55:00,425 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure (testCompactionWithHFileLink) in zk 2023-11-28 03:55:00,429 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-28 03:55:00,429 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:00,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:00,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:55:00,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:55:00,429 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-28 03:55:00,429 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testCompactionWithHFileLink' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-28 03:55:00,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:55:00,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:55:00,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-11-28 03:55:00,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:00,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:55:00,431 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure 'testCompactionWithHFileLink' on coordinator 2023-11-28 03:55:00,431 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7e7ee4c5[Count = 0] remaining members to acquire global barrier 2023-11-28 03:55:00,431 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testCompactionWithHFileLink' starting 'in-barrier' execution. 2023-11-28 03:55:00,431 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-28 03:55:00,432 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-28 03:55:00,432 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-28 03:55:00,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-28 03:55:00,432 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testCompactionWithHFileLink' received 'reached' from coordinator. 2023-11-28 03:55:00,432 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:00,432 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-28 03:55:00,432 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testCompactionWithHFileLink' locally completed 2023-11-28 03:55:00,432 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testCompactionWithHFileLink' completed for member 'jenkins-hbase4.apache.org,41203,1701143600653' in zk 2023-11-28 03:55:00,434 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:00,434 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testCompactionWithHFileLink' has notified controller of completion 2023-11-28 03:55:00,434 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:55:00,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:00,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:55:00,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:55:00,434 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testCompactionWithHFileLink' completed. 2023-11-28 03:55:00,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:55:00,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:55:00,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-11-28 03:55:00,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:00,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:55:00,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-11-28 03:55:00,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:00,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testCompactionWithHFileLink' member 'jenkins-hbase4.apache.org,41203,1701143600653': 2023-11-28 03:55:00,436 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' released barrier for procedure'testCompactionWithHFileLink', counting down latch. Waiting for 0 more 2023-11-28 03:55:00,436 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testCompactionWithHFileLink' execution completed 2023-11-28 03:55:00,436 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-28 03:55:00,436 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-28 03:55:00,436 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testCompactionWithHFileLink 2023-11-28 03:55:00,436 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testCompactionWithHFileLinkincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-28 03:55:00,438 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-28 03:55:00,438 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-28 03:55:00,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-28 03:55:00,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:55:00,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:55:00,438 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-28 03:55:00,438 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:55:00,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-28 03:55:00,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:55:00,438 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:00,438 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:55:00,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:55:00,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-11-28 03:55:00,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-28 03:55:00,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:55:00,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-11-28 03:55:00,440 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:00,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:00,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:55:00,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-11-28 03:55:00,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:00,443 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:00,443 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:55:00,443 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-11-28 03:55:00,443 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:55:00,443 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:55:00,443 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-11-28 03:55:00,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:55:00,443 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testCompactionWithHFileLink' 2023-11-28 03:55:00,444 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-28 03:55:00,443 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:55:00,443 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:00,444 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(399): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink because node does not exist (not an error) 2023-11-28 03:55:00,444 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-28 03:55:00,444 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-28 03:55:00,444 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:55:00,444 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-28 03:55:00,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:55:00,444 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testCompactionWithHFileLink'' to complete. (max 300000 ms per retry) 2023-11-28 03:55:00,444 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-28 03:55:00,448 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-11-28 03:55:00,544 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-28 03:55:00,545 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-28 03:55:00,547 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,,1701143695226.3d7f49de72f2e9e741f04714c6a7648c. with WAL disabled. Data may be lost in the event of a crash. 2023-11-28 03:55:00,547 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,1,1701143695226.b3c2fdb68f55a3cbe8d29ee7b6ddd5e5. with WAL disabled. Data may be lost in the event of a crash. 2023-11-28 03:55:00,547 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,2,1701143695226.2aad05dc9c73dcbea66259e5c0a6958f. with WAL disabled. Data may be lost in the event of a crash. 2023-11-28 03:55:00,550 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-28 03:55:00,551 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] procedure.ProcedureCoordinator(143): Procedure testCompactionWithHFileLink was in running list but was completed. Accepting new attempt. 2023-11-28 03:55:00,552 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] procedure.ProcedureCoordinator(165): Submitting procedure testCompactionWithHFileLink 2023-11-28 03:55:00,552 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testCompactionWithHFileLink' 2023-11-28 03:55:00,552 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:55:00,552 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testCompactionWithHFileLink' starting 'acquire' 2023-11-28 03:55:00,552 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testCompactionWithHFileLink', kicking off acquire phase on members. 2023-11-28 03:55:00,552 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-28 03:55:00,552 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-11-28 03:55:00,554 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:00,554 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:55:00,554 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:55:00,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:55:00,554 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:00,554 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-28 03:55:00,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-11-28 03:55:00,554 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-28 03:55:00,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-28 03:55:00,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-11-28 03:55:00,555 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testCompactionWithHFileLink 2023-11-28 03:55:00,555 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testCompactionWithHFileLink is still present, removing 2023-11-28 03:55:00,555 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testCompactionWithHFileLink 2023-11-28 03:55:00,555 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testCompactionWithHFileLink' with timeout 60000ms 2023-11-28 03:55:00,555 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:55:00,555 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testCompactionWithHFileLink' starting 'acquire' stage 2023-11-28 03:55:00,556 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,,1701143695226.3d7f49de72f2e9e741f04714c6a7648c. 2023-11-28 03:55:00,556 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,1,1701143695226.b3c2fdb68f55a3cbe8d29ee7b6ddd5e5. 2023-11-28 03:55:00,556 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-11-28 03:55:00,556 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,2,1701143695226.2aad05dc9c73dcbea66259e5c0a6958f. 2023-11-28 03:55:00,556 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,2,1701143695226.2aad05dc9c73dcbea66259e5c0a6958f. started... 2023-11-28 03:55:00,556 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,1,1701143695226.b3c2fdb68f55a3cbe8d29ee7b6ddd5e5. started... 2023-11-28 03:55:00,556 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,,1701143695226.3d7f49de72f2e9e741f04714c6a7648c. started... 2023-11-28 03:55:00,556 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-28 03:55:00,556 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 3d7f49de72f2e9e741f04714c6a7648c 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-11-28 03:55:00,556 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing b3c2fdb68f55a3cbe8d29ee7b6ddd5e5 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-11-28 03:55:00,556 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 2aad05dc9c73dcbea66259e5c0a6958f 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-11-28 03:55:00,586 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023112878e8d44c7cc44d39bafe2620db40b1d7 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023112878e8d44c7cc44d39bafe2620db40b1d7 2023-11-28 03:55:00,591 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/.tmp/family1/dc7b41ef849648dda6e80bfb0cdeea0f 2023-11-28 03:55:00,983 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20231128548ceb87df594da68e7f4d2121a8bbf7 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20231128548ceb87df594da68e7f4d2121a8bbf7 2023-11-28 03:55:00,983 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202311285c06394279144f2aa2ca5a78e6ddb299 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202311285c06394279144f2aa2ca5a78e6ddb299 2023-11-28 03:55:01,004 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202311282ac64a54290b4a59b37205d583268af5 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202311282ac64a54290b4a59b37205d583268af5 2023-11-28 03:55:01,392 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/.tmp/family1/b313b9e7ed474d04bf592f578c1928e8 2023-11-28 03:55:01,393 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/.tmp/family1/55b0944f49f248fd87ca616d0ed03e14 2023-11-28 03:55:01,408 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/.tmp/family2/b06633c919b549ed9628414e3a0d7c29 2023-11-28 03:55:01,412 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/.tmp/family1/dc7b41ef849648dda6e80bfb0cdeea0f as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family1/dc7b41ef849648dda6e80bfb0cdeea0f 2023-11-28 03:55:01,416 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family1/dc7b41ef849648dda6e80bfb0cdeea0f, entries=4, sequenceid=13, filesize=5.4 K 2023-11-28 03:55:01,417 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/.tmp/family2/b06633c919b549ed9628414e3a0d7c29 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family2/b06633c919b549ed9628414e3a0d7c29 2023-11-28 03:55:01,420 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family2/b06633c919b549ed9628414e3a0d7c29, entries=2, sequenceid=13, filesize=5.1 K 2023-11-28 03:55:01,421 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for b3c2fdb68f55a3cbe8d29ee7b6ddd5e5 in 865ms, sequenceid=13, compaction requested=true 2023-11-28 03:55:01,421 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for b3c2fdb68f55a3cbe8d29ee7b6ddd5e5: 2023-11-28 03:55:01,422 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,1,1701143695226.b3c2fdb68f55a3cbe8d29ee7b6ddd5e5. 2023-11-28 03:55:01,422 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-11-28 03:55:01,807 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202311282cafb2c5fdbd4fcf9ce84de6f28bfe1b to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202311282cafb2c5fdbd4fcf9ce84de6f28bfe1b 2023-11-28 03:55:01,809 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20231128f02c18cd45f44d048d76917a7a311c0a to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20231128f02c18cd45f44d048d76917a7a311c0a 2023-11-28 03:55:02,216 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/.tmp/family2/9d803becce0f4ac4ad8171434ff9b0a2 2023-11-28 03:55:02,216 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/.tmp/family2/cf1816b4243e4ad197a1721e4235af5a 2023-11-28 03:55:02,220 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/.tmp/family1/b313b9e7ed474d04bf592f578c1928e8 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family1/b313b9e7ed474d04bf592f578c1928e8 2023-11-28 03:55:02,220 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/.tmp/family1/55b0944f49f248fd87ca616d0ed03e14 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family1/55b0944f49f248fd87ca616d0ed03e14 2023-11-28 03:55:02,224 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family1/b313b9e7ed474d04bf592f578c1928e8, entries=4, sequenceid=13, filesize=5.4 K 2023-11-28 03:55:02,225 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family1/55b0944f49f248fd87ca616d0ed03e14, entries=4, sequenceid=13, filesize=5.4 K 2023-11-28 03:55:02,225 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/.tmp/family2/cf1816b4243e4ad197a1721e4235af5a as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family2/cf1816b4243e4ad197a1721e4235af5a 2023-11-28 03:55:02,226 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/.tmp/family2/9d803becce0f4ac4ad8171434ff9b0a2 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family2/9d803becce0f4ac4ad8171434ff9b0a2 2023-11-28 03:55:02,229 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family2/cf1816b4243e4ad197a1721e4235af5a, entries=2, sequenceid=13, filesize=5.1 K 2023-11-28 03:55:02,230 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family2/9d803becce0f4ac4ad8171434ff9b0a2, entries=2, sequenceid=13, filesize=5.1 K 2023-11-28 03:55:02,230 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 3d7f49de72f2e9e741f04714c6a7648c in 1674ms, sequenceid=13, compaction requested=true 2023-11-28 03:55:02,230 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 2aad05dc9c73dcbea66259e5c0a6958f in 1674ms, sequenceid=13, compaction requested=true 2023-11-28 03:55:02,230 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 3d7f49de72f2e9e741f04714c6a7648c: 2023-11-28 03:55:02,230 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 2aad05dc9c73dcbea66259e5c0a6958f: 2023-11-28 03:55:02,230 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,,1701143695226.3d7f49de72f2e9e741f04714c6a7648c. 2023-11-28 03:55:02,230 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,2,1701143695226.2aad05dc9c73dcbea66259e5c0a6958f. 2023-11-28 03:55:02,230 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-11-28 03:55:02,230 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-11-28 03:55:02,230 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-11-28 03:55:02,230 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:02,230 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testCompactionWithHFileLink' locally acquired 2023-11-28 03:55:02,231 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure (testCompactionWithHFileLink) in zk 2023-11-28 03:55:02,232 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:02,232 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-28 03:55:02,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:02,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:55:02,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:55:02,233 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-28 03:55:02,233 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testCompactionWithHFileLink' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-28 03:55:02,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:55:02,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:55:02,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-11-28 03:55:02,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:02,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:55:02,234 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure 'testCompactionWithHFileLink' on coordinator 2023-11-28 03:55:02,234 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5acc5b0f[Count = 0] remaining members to acquire global barrier 2023-11-28 03:55:02,234 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testCompactionWithHFileLink' starting 'in-barrier' execution. 2023-11-28 03:55:02,234 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-28 03:55:02,235 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-28 03:55:02,235 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-28 03:55:02,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-28 03:55:02,235 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testCompactionWithHFileLink' received 'reached' from coordinator. 2023-11-28 03:55:02,235 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testCompactionWithHFileLink' locally completed 2023-11-28 03:55:02,235 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:02,235 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testCompactionWithHFileLink' completed for member 'jenkins-hbase4.apache.org,41203,1701143600653' in zk 2023-11-28 03:55:02,236 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-28 03:55:02,238 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:02,238 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testCompactionWithHFileLink' has notified controller of completion 2023-11-28 03:55:02,238 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:55:02,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:02,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:55:02,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:55:02,238 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testCompactionWithHFileLink' completed. 2023-11-28 03:55:02,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:55:02,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:55:02,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-11-28 03:55:02,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:02,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:55:02,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-11-28 03:55:02,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:02,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testCompactionWithHFileLink' member 'jenkins-hbase4.apache.org,41203,1701143600653': 2023-11-28 03:55:02,241 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' released barrier for procedure'testCompactionWithHFileLink', counting down latch. Waiting for 0 more 2023-11-28 03:55:02,241 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testCompactionWithHFileLink' execution completed 2023-11-28 03:55:02,241 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-28 03:55:02,241 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-28 03:55:02,241 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testCompactionWithHFileLink 2023-11-28 03:55:02,241 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testCompactionWithHFileLinkincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-28 03:55:02,242 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-28 03:55:02,242 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-28 03:55:02,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-28 03:55:02,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:55:02,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:55:02,242 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:55:02,242 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-28 03:55:02,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-28 03:55:02,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:55:02,243 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:02,243 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:55:02,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:55:02,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-11-28 03:55:02,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-28 03:55:02,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:55:02,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-11-28 03:55:02,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:02,244 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:02,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:55:02,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-11-28 03:55:02,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:02,247 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:02,247 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:55:02,247 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-11-28 03:55:02,247 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:55:02,247 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-11-28 03:55:02,247 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:02,247 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:55:02,247 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:55:02,247 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testCompactionWithHFileLink' 2023-11-28 03:55:02,247 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-28 03:55:02,247 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-28 03:55:02,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:55:02,247 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-28 03:55:02,247 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-28 03:55:02,247 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testCompactionWithHFileLink'' to complete. (max 300000 ms per retry) 2023-11-28 03:55:02,247 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-28 03:55:02,247 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:55:02,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:55:02,348 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-28 03:55:02,349 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-28 03:55:02,350 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,,1701143695226.3d7f49de72f2e9e741f04714c6a7648c. with WAL disabled. Data may be lost in the event of a crash. 2023-11-28 03:55:02,350 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,1,1701143695226.b3c2fdb68f55a3cbe8d29ee7b6ddd5e5. with WAL disabled. Data may be lost in the event of a crash. 2023-11-28 03:55:02,350 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,2,1701143695226.2aad05dc9c73dcbea66259e5c0a6958f. with WAL disabled. Data may be lost in the event of a crash. 2023-11-28 03:55:02,352 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-28 03:55:02,354 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] procedure.ProcedureCoordinator(143): Procedure testCompactionWithHFileLink was in running list but was completed. Accepting new attempt. 2023-11-28 03:55:02,355 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] procedure.ProcedureCoordinator(165): Submitting procedure testCompactionWithHFileLink 2023-11-28 03:55:02,355 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testCompactionWithHFileLink' 2023-11-28 03:55:02,355 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:55:02,355 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testCompactionWithHFileLink' starting 'acquire' 2023-11-28 03:55:02,355 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testCompactionWithHFileLink', kicking off acquire phase on members. 2023-11-28 03:55:02,355 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-28 03:55:02,355 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-11-28 03:55:02,357 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:02,357 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:55:02,357 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:55:02,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:55:02,357 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:02,357 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-28 03:55:02,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-11-28 03:55:02,358 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-28 03:55:02,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-28 03:55:02,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-11-28 03:55:02,358 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testCompactionWithHFileLink 2023-11-28 03:55:02,358 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testCompactionWithHFileLink is still present, removing 2023-11-28 03:55:02,358 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testCompactionWithHFileLink 2023-11-28 03:55:02,358 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testCompactionWithHFileLink' with timeout 60000ms 2023-11-28 03:55:02,359 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:55:02,359 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testCompactionWithHFileLink' starting 'acquire' stage 2023-11-28 03:55:02,359 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,,1701143695226.3d7f49de72f2e9e741f04714c6a7648c. 2023-11-28 03:55:02,359 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,,1701143695226.3d7f49de72f2e9e741f04714c6a7648c. started... 2023-11-28 03:55:02,359 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,1,1701143695226.b3c2fdb68f55a3cbe8d29ee7b6ddd5e5. 2023-11-28 03:55:02,359 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 3d7f49de72f2e9e741f04714c6a7648c 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-11-28 03:55:02,359 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,1,1701143695226.b3c2fdb68f55a3cbe8d29ee7b6ddd5e5. started... 2023-11-28 03:55:02,359 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing b3c2fdb68f55a3cbe8d29ee7b6ddd5e5 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-11-28 03:55:02,361 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-11-28 03:55:02,361 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-28 03:55:02,361 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,2,1701143695226.2aad05dc9c73dcbea66259e5c0a6958f. 2023-11-28 03:55:02,361 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,2,1701143695226.2aad05dc9c73dcbea66259e5c0a6958f. started... 2023-11-28 03:55:02,361 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 2aad05dc9c73dcbea66259e5c0a6958f 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-11-28 03:55:02,777 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202311280eef9bc4c106435986d8700c2b7868aa to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202311280eef9bc4c106435986d8700c2b7868aa 2023-11-28 03:55:02,777 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20231128eb89ca1113e144ddaad3b48ac062b47f to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20231128eb89ca1113e144ddaad3b48ac062b47f 2023-11-28 03:55:02,786 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20231128bd3184bcc4744bf591f18e1d0b9b0eb7 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20231128bd3184bcc4744bf591f18e1d0b9b0eb7 2023-11-28 03:55:03,186 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/.tmp/family1/0f49b13a3dc94f1bb9ed692753ce2e94 2023-11-28 03:55:03,191 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/.tmp/family1/ab034fef3fd24871aeeb0dc81dfe66c2 2023-11-28 03:55:03,191 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/.tmp/family1/34bdbacde3fd4331ae0cc21041140fd9 2023-11-28 03:55:03,213 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023112889097115e6d245cc97ff3c92d83f537e to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023112889097115e6d245cc97ff3c92d83f537e 2023-11-28 03:55:03,214 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023112855e11b7d60d0483689ffc29e4b1a402e to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b2023112855e11b7d60d0483689ffc29e4b1a402e 2023-11-28 03:55:03,606 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202311281197d88dc8b0498eb25fac89897a29de to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202311281197d88dc8b0498eb25fac89897a29de 2023-11-28 03:55:03,618 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/.tmp/family2/cdc6a50b83e147878684e9d41c01c3e6 2023-11-28 03:55:03,619 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/.tmp/family2/fe11679925e74afa9cd10ab81215b48a 2023-11-28 03:55:03,623 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/.tmp/family1/ab034fef3fd24871aeeb0dc81dfe66c2 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family1/ab034fef3fd24871aeeb0dc81dfe66c2 2023-11-28 03:55:03,623 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/.tmp/family1/34bdbacde3fd4331ae0cc21041140fd9 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family1/34bdbacde3fd4331ae0cc21041140fd9 2023-11-28 03:55:03,627 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family1/ab034fef3fd24871aeeb0dc81dfe66c2, entries=4, sequenceid=17, filesize=5.4 K 2023-11-28 03:55:03,627 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family1/34bdbacde3fd4331ae0cc21041140fd9, entries=4, sequenceid=17, filesize=5.4 K 2023-11-28 03:55:03,627 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/.tmp/family2/cdc6a50b83e147878684e9d41c01c3e6 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family2/cdc6a50b83e147878684e9d41c01c3e6 2023-11-28 03:55:03,627 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/.tmp/family2/fe11679925e74afa9cd10ab81215b48a as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family2/fe11679925e74afa9cd10ab81215b48a 2023-11-28 03:55:03,631 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family2/fe11679925e74afa9cd10ab81215b48a, entries=2, sequenceid=17, filesize=5.1 K 2023-11-28 03:55:03,632 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for b3c2fdb68f55a3cbe8d29ee7b6ddd5e5 in 1273ms, sequenceid=17, compaction requested=true 2023-11-28 03:55:03,632 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for b3c2fdb68f55a3cbe8d29ee7b6ddd5e5: 2023-11-28 03:55:03,632 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,1,1701143695226.b3c2fdb68f55a3cbe8d29ee7b6ddd5e5. 2023-11-28 03:55:03,632 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-11-28 03:55:03,633 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family2/cdc6a50b83e147878684e9d41c01c3e6, entries=2, sequenceid=17, filesize=5.1 K 2023-11-28 03:55:03,634 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 2aad05dc9c73dcbea66259e5c0a6958f in 1273ms, sequenceid=17, compaction requested=true 2023-11-28 03:55:03,634 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 2aad05dc9c73dcbea66259e5c0a6958f: 2023-11-28 03:55:03,634 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,2,1701143695226.2aad05dc9c73dcbea66259e5c0a6958f. 2023-11-28 03:55:03,634 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-11-28 03:55:04,012 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/.tmp/family2/a7b3c563e013482e86b9f3270fa9799c 2023-11-28 03:55:04,017 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/.tmp/family1/0f49b13a3dc94f1bb9ed692753ce2e94 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family1/0f49b13a3dc94f1bb9ed692753ce2e94 2023-11-28 03:55:04,021 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family1/0f49b13a3dc94f1bb9ed692753ce2e94, entries=4, sequenceid=17, filesize=5.4 K 2023-11-28 03:55:04,021 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/.tmp/family2/a7b3c563e013482e86b9f3270fa9799c as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family2/a7b3c563e013482e86b9f3270fa9799c 2023-11-28 03:55:04,025 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family2/a7b3c563e013482e86b9f3270fa9799c, entries=2, sequenceid=17, filesize=5.1 K 2023-11-28 03:55:04,026 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 3d7f49de72f2e9e741f04714c6a7648c in 1667ms, sequenceid=17, compaction requested=true 2023-11-28 03:55:04,026 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 3d7f49de72f2e9e741f04714c6a7648c: 2023-11-28 03:55:04,026 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,,1701143695226.3d7f49de72f2e9e741f04714c6a7648c. 2023-11-28 03:55:04,026 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-11-28 03:55:04,026 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-11-28 03:55:04,026 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:04,027 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testCompactionWithHFileLink' locally acquired 2023-11-28 03:55:04,027 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure (testCompactionWithHFileLink) in zk 2023-11-28 03:55:04,029 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:04,029 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-28 03:55:04,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:04,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:55:04,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:55:04,030 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-28 03:55:04,030 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testCompactionWithHFileLink' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-28 03:55:04,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:55:04,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:55:04,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-11-28 03:55:04,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:04,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:55:04,031 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure 'testCompactionWithHFileLink' on coordinator 2023-11-28 03:55:04,031 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1d328289[Count = 0] remaining members to acquire global barrier 2023-11-28 03:55:04,031 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testCompactionWithHFileLink' starting 'in-barrier' execution. 2023-11-28 03:55:04,031 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-28 03:55:04,032 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-28 03:55:04,032 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-28 03:55:04,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-28 03:55:04,032 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testCompactionWithHFileLink' received 'reached' from coordinator. 2023-11-28 03:55:04,032 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testCompactionWithHFileLink' locally completed 2023-11-28 03:55:04,032 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testCompactionWithHFileLink' completed for member 'jenkins-hbase4.apache.org,41203,1701143600653' in zk 2023-11-28 03:55:04,033 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:04,033 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-28 03:55:04,034 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:04,034 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testCompactionWithHFileLink' has notified controller of completion 2023-11-28 03:55:04,034 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:55:04,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:04,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:55:04,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:55:04,034 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testCompactionWithHFileLink' completed. 2023-11-28 03:55:04,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:55:04,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:55:04,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-11-28 03:55:04,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:04,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:55:04,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-11-28 03:55:04,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:04,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testCompactionWithHFileLink' member 'jenkins-hbase4.apache.org,41203,1701143600653': 2023-11-28 03:55:04,037 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testCompactionWithHFileLink' execution completed 2023-11-28 03:55:04,037 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-28 03:55:04,037 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-28 03:55:04,037 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testCompactionWithHFileLink 2023-11-28 03:55:04,037 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testCompactionWithHFileLinkincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-28 03:55:04,037 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' released barrier for procedure'testCompactionWithHFileLink', counting down latch. Waiting for 0 more 2023-11-28 03:55:04,038 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-28 03:55:04,038 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:55:04,038 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-28 03:55:04,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-28 03:55:04,038 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-28 03:55:04,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-28 03:55:04,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:55:04,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:55:04,039 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:55:04,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:55:04,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:55:04,039 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:04,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-28 03:55:04,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-11-28 03:55:04,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:55:04,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-11-28 03:55:04,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:04,040 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:04,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:55:04,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-11-28 03:55:04,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:04,044 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:04,044 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:55:04,044 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-11-28 03:55:04,044 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testCompactionWithHFileLink' 2023-11-28 03:55:04,044 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-28 03:55:04,044 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:55:04,044 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:55:04,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:55:04,044 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:55:04,044 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-11-28 03:55:04,044 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:04,044 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-28 03:55:04,044 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testCompactionWithHFileLink'' to complete. (max 300000 ms per retry) 2023-11-28 03:55:04,044 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-28 03:55:04,044 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-28 03:55:04,045 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-28 03:55:04,045 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:55:04,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:55:04,145 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-28 03:55:04,146 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-28 03:55:04,156 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(1583): Client=jenkins//172.31.14.131 snapshot request for:{ ss=snaptb-1701143704146 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-11-28 03:55:04,156 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] snapshot.SnapshotDescriptionUtils(316): Creation time not specified, setting to:1701143704156 (current time:1701143704156). 2023-11-28 03:55:04,156 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] snapshot.SnapshotDescriptionUtils(332): Snapshot current TTL value: 0 resetting it to default value: 0 2023-11-28 03:55:04,157 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] zookeeper.ReadOnlyZKClient(139): Connect 0x10e7648b to 127.0.0.1:55641 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-11-28 03:55:04,164 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] ipc.AbstractRpcClient(189): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@555d0903, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-11-28 03:55:04,168 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] ipc.RpcConnection(122): Using SIMPLE authentication for service=ClientService, sasl=false 2023-11-28 03:55:04,170 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:45556, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-11-28 03:55:04,171 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x10e7648b to 127.0.0.1:55641 2023-11-28 03:55:04,171 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] ipc.AbstractRpcClient(489): Stopping rpc client 2023-11-28 03:55:04,171 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] snapshot.SnapshotManager(637): No existing snapshot, attempting snapshot... 2023-11-28 03:55:04,172 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] snapshot.SnapshotManager(684): Table enabled, starting distributed snapshots for { ss=snaptb-1701143704146 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-11-28 03:55:04,184 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] snapshot.SnapshotManager(519): Unable to retrieve ACL status for path: hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/.hbase-snapshot, current working dir path: hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/.hbase-snapshot/.tmp/snaptb-1701143704146 org.apache.hadoop.hdfs.protocol.AclException: The ACL operation has been rejected. Support for ACLs has been disabled by setting dfs.namenode.acls.enabled to false. at org.apache.hadoop.hdfs.server.namenode.FSDirAclOp.checkAclsConfigFlag(FSDirAclOp.java:197) at org.apache.hadoop.hdfs.server.namenode.FSDirAclOp.getAclStatus(FSDirAclOp.java:150) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAclStatus(FSNamesystem.java:6744) at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.getAclStatus(NameNodeRpcServer.java:2065) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.getAclStatus(ClientNamenodeProtocolServerSideTranslatorPB.java:1378) at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java) at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:507) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1034) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:994) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:922) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1893) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2833) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:121) at org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:88) at org.apache.hadoop.hdfs.DFSClient.getAclStatus(DFSClient.java:2736) at org.apache.hadoop.hdfs.DistributedFileSystem$51.doCall(DistributedFileSystem.java:2294) at org.apache.hadoop.hdfs.DistributedFileSystem$51.doCall(DistributedFileSystem.java:2291) at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81) at org.apache.hadoop.hdfs.DistributedFileSystem.getAclStatus(DistributedFileSystem.java:2291) at org.apache.hadoop.hbase.master.snapshot.SnapshotManager.updateWorkingDirAclsIfRequired(SnapshotManager.java:517) at org.apache.hadoop.hbase.master.snapshot.SnapshotManager.prepareToTakeSnapshot(SnapshotManager.java:490) at org.apache.hadoop.hbase.master.snapshot.SnapshotManager.snapshotEnabledTable(SnapshotManager.java:559) at org.apache.hadoop.hbase.master.snapshot.SnapshotManager.takeSnapshotInternal(SnapshotManager.java:687) at org.apache.hadoop.hbase.master.snapshot.SnapshotManager.takeSnapshot(SnapshotManager.java:623) at org.apache.hadoop.hbase.master.MasterRpcServices.snapshot(MasterRpcServices.java:1588) at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) Caused by: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.hdfs.protocol.AclException): The ACL operation has been rejected. Support for ACLs has been disabled by setting dfs.namenode.acls.enabled to false. at org.apache.hadoop.hdfs.server.namenode.FSDirAclOp.checkAclsConfigFlag(FSDirAclOp.java:197) at org.apache.hadoop.hdfs.server.namenode.FSDirAclOp.getAclStatus(FSDirAclOp.java:150) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAclStatus(FSNamesystem.java:6744) at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.getAclStatus(NameNodeRpcServer.java:2065) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.getAclStatus(ClientNamenodeProtocolServerSideTranslatorPB.java:1378) at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java) at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:507) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1034) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:994) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:922) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1893) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2833) at org.apache.hadoop.ipc.Client.getRpcResponse(Client.java:1540) at org.apache.hadoop.ipc.Client.call(Client.java:1486) at org.apache.hadoop.ipc.Client.call(Client.java:1385) at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:232) at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:118) at com.sun.proxy.$Proxy29.getAclStatus(Unknown Source) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getAclStatus(ClientNamenodeProtocolTranslatorPB.java:1385) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:422) at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeMethod(RetryInvocationHandler.java:165) at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invoke(RetryInvocationHandler.java:157) at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeOnce(RetryInvocationHandler.java:95) at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:359) at com.sun.proxy.$Proxy30.getAclStatus(Unknown Source) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.getAclStatus(Unknown Source) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.getAclStatus(Unknown Source) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.getAclStatus(Unknown Source) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.getAclStatus(Unknown Source) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.getAclStatus(Unknown Source) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.getAclStatus(Unknown Source) at org.apache.hadoop.hdfs.DFSClient.getAclStatus(DFSClient.java:2734) ... 15 more 2023-11-28 03:55:04,191 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] procedure2.ProcedureExecutor(1028): Stored pid=86, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-11-28 03:55:04,192 DEBUG [PEWorker-2] locking.LockProcedure(309): LOCKED pid=86, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-11-28 03:55:04,193 INFO [PEWorker-2] procedure2.TimeoutExecutorThread(81): ADDED pid=86, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE; timeout=600000, timestamp=1701144304193 2023-11-28 03:55:04,193 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] snapshot.SnapshotManager(689): Started snapshot: { ss=snaptb-1701143704146 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-11-28 03:55:04,193 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(174): Running FLUSH table snapshot snaptb-1701143704146 C_M_SNAPSHOT_TABLE on table testCompactionWithHFileLink 2023-11-28 03:55:04,194 DEBUG [PEWorker-1] locking.LockProcedure(242): UNLOCKED pid=86, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-11-28 03:55:04,195 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] procedure2.ProcedureExecutor(1028): Stored pid=87, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=SHARED 2023-11-28 03:55:04,196 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=86, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE in 3 msec 2023-11-28 03:55:04,196 DEBUG [PEWorker-1] locking.LockProcedure(309): LOCKED pid=87, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=SHARED 2023-11-28 03:55:04,197 INFO [PEWorker-1] procedure2.TimeoutExecutorThread(81): ADDED pid=87, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=SHARED; timeout=600000, timestamp=1701144304197 2023-11-28 03:55:04,199 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2418): Waiting a max of 300000 ms for snapshot '{ ss=snaptb-1701143704146 table=testCompactionWithHFileLink type=FLUSH ttl=0 }'' to complete. (max 300000 ms per retry) 2023-11-28 03:55:04,199 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2428): (#1) Sleeping: 100ms while waiting for snapshot completion. 2023-11-28 03:55:04,203 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] procedure.ProcedureCoordinator(165): Submitting procedure snaptb-1701143704146 2023-11-28 03:55:04,204 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'snaptb-1701143704146' 2023-11-28 03:55:04,204 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 300000 ms 2023-11-28 03:55:04,204 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'snaptb-1701143704146' starting 'acquire' 2023-11-28 03:55:04,204 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'snaptb-1701143704146', kicking off acquire phase on members. 2023-11-28 03:55:04,204 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/abort/snaptb-1701143704146 2023-11-28 03:55:04,204 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/online-snapshot/acquired/snaptb-1701143704146 2023-11-28 03:55:04,206 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/online-snapshot/acquired/snaptb-1701143704146/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:04,206 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/acquired 2023-11-28 03:55:04,207 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/online-snapshot/acquired 2023-11-28 03:55:04,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-11-28 03:55:04,207 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/acquired/snaptb-1701143704146/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:04,207 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-28 03:55:04,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/online-snapshot/acquired/snaptb-1701143704146 2023-11-28 03:55:04,207 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/abort/snaptb-1701143704146 2023-11-28 03:55:04,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 68 2023-11-28 03:55:04,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/online-snapshot/acquired/snaptb-1701143704146 2023-11-28 03:55:04,208 DEBUG [zk-event-processor-pool-0] snapshot.RegionServerSnapshotManager(175): Launching subprocedure for snapshot snaptb-1701143704146 from table testCompactionWithHFileLink type FLUSH 2023-11-28 03:55:04,211 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:snaptb-1701143704146 2023-11-28 03:55:04,211 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'snaptb-1701143704146' with timeout 300000ms 2023-11-28 03:55:04,211 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 300000 ms 2023-11-28 03:55:04,211 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'snaptb-1701143704146' starting 'acquire' stage 2023-11-28 03:55:04,211 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'snaptb-1701143704146' locally acquired 2023-11-28 03:55:04,211 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure (snaptb-1701143704146) in zk 2023-11-28 03:55:04,213 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/online-snapshot/reached/snaptb-1701143704146 2023-11-28 03:55:04,213 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/acquired/snaptb-1701143704146/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:04,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/online-snapshot/acquired/snaptb-1701143704146/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:04,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:55:04,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/online-snapshot 2023-11-28 03:55:04,214 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/reached/snaptb-1701143704146 2023-11-28 03:55:04,214 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'snaptb-1701143704146' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-28 03:55:04,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:55:04,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:55:04,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1701143704146 2023-11-28 03:55:04,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:04,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:55:04,215 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure 'snaptb-1701143704146' on coordinator 2023-11-28 03:55:04,215 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2535e921[Count = 0] remaining members to acquire global barrier 2023-11-28 03:55:04,215 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'snaptb-1701143704146' starting 'in-barrier' execution. 2023-11-28 03:55:04,215 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/online-snapshot/reached/snaptb-1701143704146 2023-11-28 03:55:04,216 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/reached/snaptb-1701143704146 2023-11-28 03:55:04,216 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/online-snapshot/reached/snaptb-1701143704146 2023-11-28 03:55:04,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/online-snapshot/reached/snaptb-1701143704146 2023-11-28 03:55:04,216 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'snaptb-1701143704146' received 'reached' from coordinator. 2023-11-28 03:55:04,217 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/reached/snaptb-1701143704146/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:04,217 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-28 03:55:04,217 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(97): Starting snapshot operation on testCompactionWithHFileLink,,1701143695226.3d7f49de72f2e9e741f04714c6a7648c. 2023-11-28 03:55:04,218 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(110): Flush Snapshotting region testCompactionWithHFileLink,,1701143695226.3d7f49de72f2e9e741f04714c6a7648c. started... 2023-11-28 03:55:04,218 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-snapshot-pool-0] regionserver.HRegion(2446): Flush status journal for 3d7f49de72f2e9e741f04714c6a7648c: 2023-11-28 03:55:04,218 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-snapshot-pool-1] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(97): Starting snapshot operation on testCompactionWithHFileLink,1,1701143695226.b3c2fdb68f55a3cbe8d29ee7b6ddd5e5. 2023-11-28 03:55:04,218 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] snapshot.FlushSnapshotSubprocedure(170): Flush Snapshot Tasks submitted for 3 regions 2023-11-28 03:55:04,218 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-snapshot-pool-2] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(97): Starting snapshot operation on testCompactionWithHFileLink,2,1701143695226.2aad05dc9c73dcbea66259e5c0a6958f. 2023-11-28 03:55:04,218 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-snapshot-pool-1] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(110): Flush Snapshotting region testCompactionWithHFileLink,1,1701143695226.b3c2fdb68f55a3cbe8d29ee7b6ddd5e5. started... 2023-11-28 03:55:04,218 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-snapshot-pool-2] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(110): Flush Snapshotting region testCompactionWithHFileLink,2,1701143695226.2aad05dc9c73dcbea66259e5c0a6958f. started... 2023-11-28 03:55:04,218 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(301): Waiting for local region snapshots to finish. 2023-11-28 03:55:04,218 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-snapshot-pool-1] regionserver.HRegion(2446): Flush status journal for b3c2fdb68f55a3cbe8d29ee7b6ddd5e5: 2023-11-28 03:55:04,218 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-snapshot-pool-2] regionserver.HRegion(2446): Flush status journal for 2aad05dc9c73dcbea66259e5c0a6958f: 2023-11-28 03:55:04,219 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-snapshot-pool-2] snapshot.SnapshotManifest(238): Storing 'testCompactionWithHFileLink,2,1701143695226.2aad05dc9c73dcbea66259e5c0a6958f.' region-info for snapshot=snaptb-1701143704146 2023-11-28 03:55:04,219 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-snapshot-pool-1] snapshot.SnapshotManifest(238): Storing 'testCompactionWithHFileLink,1,1701143695226.b3c2fdb68f55a3cbe8d29ee7b6ddd5e5.' region-info for snapshot=snaptb-1701143704146 2023-11-28 03:55:04,219 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-snapshot-pool-0] snapshot.SnapshotManifest(238): Storing 'testCompactionWithHFileLink,,1701143695226.3d7f49de72f2e9e741f04714c6a7648c.' region-info for snapshot=snaptb-1701143704146 2023-11-28 03:55:04,224 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-snapshot-pool-1] snapshot.SnapshotManifest(243): Creating references for hfiles 2023-11-28 03:55:04,224 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-snapshot-pool-2] snapshot.SnapshotManifest(243): Creating references for hfiles 2023-11-28 03:55:04,224 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-snapshot-pool-0] snapshot.SnapshotManifest(243): Creating references for hfiles 2023-11-28 03:55:04,228 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-snapshot-pool-1] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family1/88a76a3cbca146b6bb71214ecc4a9c87, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family1/299d98e37656434ea5b2482137c58710, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family1/dc7b41ef849648dda6e80bfb0cdeea0f, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family1/34bdbacde3fd4331ae0cc21041140fd9] hfiles 2023-11-28 03:55:04,228 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-snapshot-pool-0] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family1/9cba99270b974d6b9181045f6f7ee6bc, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family1/149c75fc3d8a444bbd7d2180c2614583, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family1/b313b9e7ed474d04bf592f578c1928e8, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family1/0f49b13a3dc94f1bb9ed692753ce2e94] hfiles 2023-11-28 03:55:04,228 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family1/88a76a3cbca146b6bb71214ecc4a9c87 for snapshot=snaptb-1701143704146 2023-11-28 03:55:04,228 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family1/9cba99270b974d6b9181045f6f7ee6bc for snapshot=snaptb-1701143704146 2023-11-28 03:55:04,228 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-snapshot-pool-2] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family1/5bd4594355ca45ff98e17e8e298757d3, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family1/ef8ca9f746b4468d8da6b892ab104927, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family1/55b0944f49f248fd87ca616d0ed03e14, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family1/ab034fef3fd24871aeeb0dc81dfe66c2] hfiles 2023-11-28 03:55:04,228 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family1/5bd4594355ca45ff98e17e8e298757d3 for snapshot=snaptb-1701143704146 2023-11-28 03:55:04,231 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family1/299d98e37656434ea5b2482137c58710 for snapshot=snaptb-1701143704146 2023-11-28 03:55:04,231 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family1/ef8ca9f746b4468d8da6b892ab104927 for snapshot=snaptb-1701143704146 2023-11-28 03:55:04,231 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family1/149c75fc3d8a444bbd7d2180c2614583 for snapshot=snaptb-1701143704146 2023-11-28 03:55:04,231 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family1/55b0944f49f248fd87ca616d0ed03e14 for snapshot=snaptb-1701143704146 2023-11-28 03:55:04,231 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family1/dc7b41ef849648dda6e80bfb0cdeea0f for snapshot=snaptb-1701143704146 2023-11-28 03:55:04,231 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family1/ab034fef3fd24871aeeb0dc81dfe66c2 for snapshot=snaptb-1701143704146 2023-11-28 03:55:04,231 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family1/b313b9e7ed474d04bf592f578c1928e8 for snapshot=snaptb-1701143704146 2023-11-28 03:55:04,231 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family1/34bdbacde3fd4331ae0cc21041140fd9 for snapshot=snaptb-1701143704146 2023-11-28 03:55:04,231 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family1/0f49b13a3dc94f1bb9ed692753ce2e94 for snapshot=snaptb-1701143704146 2023-11-28 03:55:04,231 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-snapshot-pool-2] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family2/a79a53b7062d4e53b9199b02c23f6c24, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family2/a5d2b1b5f59d4745beda10e8ec8e729f, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family2/9d803becce0f4ac4ad8171434ff9b0a2, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family2/cdc6a50b83e147878684e9d41c01c3e6] hfiles 2023-11-28 03:55:04,231 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-snapshot-pool-0] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family2/cfbbdda0c47f440ab7343ce95b4baaf5, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family2/14783dff7d074f04ac3290bc0e4a92ba, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family2/cf1816b4243e4ad197a1721e4235af5a, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family2/a7b3c563e013482e86b9f3270fa9799c] hfiles 2023-11-28 03:55:04,231 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family2/a79a53b7062d4e53b9199b02c23f6c24 for snapshot=snaptb-1701143704146 2023-11-28 03:55:04,231 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-snapshot-pool-1] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family2/ee2d50b684b2479e8a02236cf5083faa, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family2/75ff1f4a28e9466e86a2f50fe4b3ba7c, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family2/b06633c919b549ed9628414e3a0d7c29, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family2/fe11679925e74afa9cd10ab81215b48a] hfiles 2023-11-28 03:55:04,232 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family2/ee2d50b684b2479e8a02236cf5083faa for snapshot=snaptb-1701143704146 2023-11-28 03:55:04,231 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family2/a5d2b1b5f59d4745beda10e8ec8e729f for snapshot=snaptb-1701143704146 2023-11-28 03:55:04,231 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family2/cfbbdda0c47f440ab7343ce95b4baaf5 for snapshot=snaptb-1701143704146 2023-11-28 03:55:04,232 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family2/9d803becce0f4ac4ad8171434ff9b0a2 for snapshot=snaptb-1701143704146 2023-11-28 03:55:04,232 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family2/75ff1f4a28e9466e86a2f50fe4b3ba7c for snapshot=snaptb-1701143704146 2023-11-28 03:55:04,232 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family2/cdc6a50b83e147878684e9d41c01c3e6 for snapshot=snaptb-1701143704146 2023-11-28 03:55:04,232 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family2/14783dff7d074f04ac3290bc0e4a92ba for snapshot=snaptb-1701143704146 2023-11-28 03:55:04,232 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family2/b06633c919b549ed9628414e3a0d7c29 for snapshot=snaptb-1701143704146 2023-11-28 03:55:04,232 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family2/cf1816b4243e4ad197a1721e4235af5a for snapshot=snaptb-1701143704146 2023-11-28 03:55:04,232 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family2/fe11679925e74afa9cd10ab81215b48a for snapshot=snaptb-1701143704146 2023-11-28 03:55:04,232 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family2/a7b3c563e013482e86b9f3270fa9799c for snapshot=snaptb-1701143704146 2023-11-28 03:55:04,299 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-11-28 03:55:04,302 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(1212): Checking to see if snapshot from request:{ ss=snaptb-1701143704146 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-11-28 03:55:04,304 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] snapshot.SnapshotManager(406): Snapshoting '{ ss=snaptb-1701143704146 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-11-28 03:55:04,305 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2428): (#2) Sleeping: 200ms while waiting for snapshot completion. 2023-11-28 03:55:04,505 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-11-28 03:55:04,506 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(1212): Checking to see if snapshot from request:{ ss=snaptb-1701143704146 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-11-28 03:55:04,506 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] snapshot.SnapshotManager(406): Snapshoting '{ ss=snaptb-1701143704146 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-11-28 03:55:04,506 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2428): (#3) Sleeping: 300ms while waiting for snapshot completion. 2023-11-28 03:55:04,643 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-snapshot-pool-1] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(137): ... Flush Snapshotting region testCompactionWithHFileLink,1,1701143695226.b3c2fdb68f55a3cbe8d29ee7b6ddd5e5. completed. 2023-11-28 03:55:04,643 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-snapshot-pool-1] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(140): Closing snapshot operation on testCompactionWithHFileLink,1,1701143695226.b3c2fdb68f55a3cbe8d29ee7b6ddd5e5. 2023-11-28 03:55:04,643 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(137): ... Flush Snapshotting region testCompactionWithHFileLink,,1701143695226.3d7f49de72f2e9e741f04714c6a7648c. completed. 2023-11-28 03:55:04,643 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(140): Closing snapshot operation on testCompactionWithHFileLink,,1701143695226.3d7f49de72f2e9e741f04714c6a7648c. 2023-11-28 03:55:04,643 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-snapshot-pool-2] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(137): ... Flush Snapshotting region testCompactionWithHFileLink,2,1701143695226.2aad05dc9c73dcbea66259e5c0a6958f. completed. 2023-11-28 03:55:04,643 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-snapshot-pool-2] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(140): Closing snapshot operation on testCompactionWithHFileLink,2,1701143695226.2aad05dc9c73dcbea66259e5c0a6958f. 2023-11-28 03:55:04,643 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(312): Completed 1/3 local region snapshots. 2023-11-28 03:55:04,643 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(312): Completed 2/3 local region snapshots. 2023-11-28 03:55:04,643 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(312): Completed 3/3 local region snapshots. 2023-11-28 03:55:04,643 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(314): Completed 3 local region snapshots. 2023-11-28 03:55:04,643 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(345): cancelling 0 tasks for snapshot jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:04,643 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'snaptb-1701143704146' locally completed 2023-11-28 03:55:04,643 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'snaptb-1701143704146' completed for member 'jenkins-hbase4.apache.org,41203,1701143600653' in zk 2023-11-28 03:55:04,645 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/reached/snaptb-1701143704146/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:04,645 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'snaptb-1701143704146' has notified controller of completion 2023-11-28 03:55:04,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/online-snapshot/reached/snaptb-1701143704146/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:04,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:55:04,645 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:55:04,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/online-snapshot 2023-11-28 03:55:04,645 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'snaptb-1701143704146' completed. 2023-11-28 03:55:04,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:55:04,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:55:04,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1701143704146 2023-11-28 03:55:04,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:04,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:55:04,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1701143704146 2023-11-28 03:55:04,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:04,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'snaptb-1701143704146' member 'jenkins-hbase4.apache.org,41203,1701143600653': 2023-11-28 03:55:04,648 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' released barrier for procedure'snaptb-1701143704146', counting down latch. Waiting for 0 more 2023-11-28 03:55:04,648 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'snaptb-1701143704146' execution completed 2023-11-28 03:55:04,648 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-28 03:55:04,648 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-28 03:55:04,648 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:snaptb-1701143704146 2023-11-28 03:55:04,648 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure snaptb-1701143704146including nodes /hbase/online-snapshot/acquired /hbase/online-snapshot/reached /hbase/online-snapshot/abort 2023-11-28 03:55:04,651 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/abort/snaptb-1701143704146 2023-11-28 03:55:04,651 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/abort/snaptb-1701143704146 2023-11-28 03:55:04,651 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/online-snapshot/abort/snaptb-1701143704146 2023-11-28 03:55:04,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/online-snapshot/abort/snaptb-1701143704146 2023-11-28 03:55:04,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/online-snapshot/abort/snaptb-1701143704146 2023-11-28 03:55:04,651 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/abort 2023-11-28 03:55:04,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:55:04,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/online-snapshot 2023-11-28 03:55:04,652 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/online-snapshot/abort 2023-11-28 03:55:04,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-11-28 03:55:04,652 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/online-snapshot/acquired/snaptb-1701143704146/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:04,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:55:04,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/online-snapshot/abort/snaptb-1701143704146 2023-11-28 03:55:04,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1701143704146 2023-11-28 03:55:04,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:55:04,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1701143704146 2023-11-28 03:55:04,653 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/online-snapshot/reached/snaptb-1701143704146/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:04,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:04,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:55:04,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1701143704146 2023-11-28 03:55:04,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:04,655 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/acquired/snaptb-1701143704146/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:04,655 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/acquired/snaptb-1701143704146 2023-11-28 03:55:04,655 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/acquired/snaptb-1701143704146 2023-11-28 03:55:04,655 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/acquired 2023-11-28 03:55:04,655 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:55:04,655 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/online-snapshot/acquired 2023-11-28 03:55:04,656 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.EnabledTableSnapshotHandler(97): Done waiting - online snapshot for snaptb-1701143704146 2023-11-28 03:55:04,655 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/abort 2023-11-28 03:55:04,656 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.EnabledTableSnapshotHandler(113): Taking snapshot for mob files in table testCompactionWithHFileLink 2023-11-28 03:55:04,655 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(399): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Unable to get data of znode /hbase/online-snapshot/abort/snaptb-1701143704146 because node does not exist (not an error) 2023-11-28 03:55:04,656 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(191): Storing mob region '{ENCODED => 0e7b6fa9e81bd62588edd7d77d6165e4, NAME => 'testCompactionWithHFileLink,.mob,0.0e7b6fa9e81bd62588edd7d77d6165e4.', STARTKEY => '.mob', ENDKEY => ''}' region-info for snapshot=snaptb-1701143704146 2023-11-28 03:55:04,655 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/reached/snaptb-1701143704146/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:04,656 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(196): Creating references for mob files 2023-11-28 03:55:04,656 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/reached/snaptb-1701143704146 2023-11-28 03:55:04,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-11-28 03:55:04,656 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/reached/snaptb-1701143704146 2023-11-28 03:55:04,656 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/abort/snaptb-1701143704146 2023-11-28 03:55:04,656 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/online-snapshot/abort 2023-11-28 03:55:04,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-11-28 03:55:04,657 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023112838c9c1bb8cde4e7f815926db3a63a195, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023112878e8d44c7cc44d39bafe2620db40b1d7, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202311287fb879a012e040c992a5c4c12e52bff2, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20231128bd3184bcc4744bf591f18e1d0b9b0eb7, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20231128548ceb87df594da68e7f4d2121a8bbf7, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20231128dec752e8fb644a0a80d6b4fd043402c1, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20231128eb89ca1113e144ddaad3b48ac062b47f, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20231128fb09572884984543b4a9faae21b5679c, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202311280eef9bc4c106435986d8700c2b7868aa, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202311285c06394279144f2aa2ca5a78e6ddb299, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e2023112864ca880632ff4b81b856e56f97a51ba5, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20231128ae21d695d78b47819031a63d23fd1adf] mob files 2023-11-28 03:55:04,657 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (1/12): hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023112838c9c1bb8cde4e7f815926db3a63a195 2023-11-28 03:55:04,657 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (2/12): hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023112878e8d44c7cc44d39bafe2620db40b1d7 2023-11-28 03:55:04,657 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (3/12): hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202311287fb879a012e040c992a5c4c12e52bff2 2023-11-28 03:55:04,657 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (4/12): hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20231128bd3184bcc4744bf591f18e1d0b9b0eb7 2023-11-28 03:55:04,657 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (5/12): hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20231128548ceb87df594da68e7f4d2121a8bbf7 2023-11-28 03:55:04,657 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (6/12): hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20231128dec752e8fb644a0a80d6b4fd043402c1 2023-11-28 03:55:04,657 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (7/12): hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20231128eb89ca1113e144ddaad3b48ac062b47f 2023-11-28 03:55:04,658 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (8/12): hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20231128fb09572884984543b4a9faae21b5679c 2023-11-28 03:55:04,658 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (9/12): hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202311280eef9bc4c106435986d8700c2b7868aa 2023-11-28 03:55:04,658 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (10/12): hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202311285c06394279144f2aa2ca5a78e6ddb299 2023-11-28 03:55:04,658 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (11/12): hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e2023112864ca880632ff4b81b856e56f97a51ba5 2023-11-28 03:55:04,658 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (12/12): hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20231128ae21d695d78b47819031a63d23fd1adf 2023-11-28 03:55:04,659 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202311282ac64a54290b4a59b37205d583268af5, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202311283312fc03dd134b7bab571a71f58de4a3, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202311284dec3a2023324667bf8796b80a2f3bce, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b2023112855e11b7d60d0483689ffc29e4b1a402e, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202311284eac4cd4063e44bf9de8ec54912c3ff0, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202311284edd51e4fb9f46e894cf420c094d1251, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023112889097115e6d245cc97ff3c92d83f537e, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20231128f02c18cd45f44d048d76917a7a311c0a, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202311281197d88dc8b0498eb25fac89897a29de, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202311282cafb2c5fdbd4fcf9ce84de6f28bfe1b, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20231128310f916e533545709f94215002f0c70b, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202311284073e83e8d4242548321ae15f6b216d4] mob files 2023-11-28 03:55:04,659 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (1/12): hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202311282ac64a54290b4a59b37205d583268af5 2023-11-28 03:55:04,659 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (2/12): hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202311283312fc03dd134b7bab571a71f58de4a3 2023-11-28 03:55:04,659 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (3/12): hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202311284dec3a2023324667bf8796b80a2f3bce 2023-11-28 03:55:04,659 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (4/12): hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b2023112855e11b7d60d0483689ffc29e4b1a402e 2023-11-28 03:55:04,659 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (5/12): hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202311284eac4cd4063e44bf9de8ec54912c3ff0 2023-11-28 03:55:04,659 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (6/12): hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202311284edd51e4fb9f46e894cf420c094d1251 2023-11-28 03:55:04,659 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (7/12): hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023112889097115e6d245cc97ff3c92d83f537e 2023-11-28 03:55:04,659 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (8/12): hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20231128f02c18cd45f44d048d76917a7a311c0a 2023-11-28 03:55:04,659 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (9/12): hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202311281197d88dc8b0498eb25fac89897a29de 2023-11-28 03:55:04,659 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (10/12): hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202311282cafb2c5fdbd4fcf9ce84de6f28bfe1b 2023-11-28 03:55:04,659 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (11/12): hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20231128310f916e533545709f94215002f0c70b 2023-11-28 03:55:04,659 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (12/12): hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202311284073e83e8d4242548321ae15f6b216d4 2023-11-28 03:55:04,806 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-11-28 03:55:04,807 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(1212): Checking to see if snapshot from request:{ ss=snaptb-1701143704146 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-11-28 03:55:04,807 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] snapshot.SnapshotManager(406): Snapshoting '{ ss=snaptb-1701143704146 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-11-28 03:55:04,807 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2428): (#4) Sleeping: 500ms while waiting for snapshot completion. 2023-11-28 03:55:05,065 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(484): Convert to Single Snapshot Manifest for snaptb-1701143704146 2023-11-28 03:55:05,067 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifestV1(126): No regions under directory:hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/.hbase-snapshot/.tmp/snaptb-1701143704146 2023-11-28 03:55:05,308 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-11-28 03:55:05,309 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(1212): Checking to see if snapshot from request:{ ss=snaptb-1701143704146 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-11-28 03:55:05,309 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] snapshot.SnapshotManager(406): Snapshoting '{ ss=snaptb-1701143704146 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-11-28 03:55:05,309 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2428): (#5) Sleeping: 1000ms while waiting for snapshot completion. 2023-11-28 03:55:05,527 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotDescriptionUtils(404): Sentinel is done, just moving the snapshot from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/.hbase-snapshot/.tmp/snaptb-1701143704146 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/.hbase-snapshot/snaptb-1701143704146 2023-11-28 03:55:05,528 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(229): Snapshot snaptb-1701143704146 of table testCompactionWithHFileLink completed 2023-11-28 03:55:05,528 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(246): Launching cleanup of working dir:hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/.hbase-snapshot/.tmp/snaptb-1701143704146 2023-11-28 03:55:05,529 ERROR [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(251): Couldn't delete snapshot working directory:hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/.hbase-snapshot/.tmp/snaptb-1701143704146 2023-11-28 03:55:05,529 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(257): Table snapshot journal : Running FLUSH table snapshot snaptb-1701143704146 C_M_SNAPSHOT_TABLE on table testCompactionWithHFileLink at 1701143704193Completed referencing HFiles for the mob region of table: testCompactionWithHFileLink at 1701143705064 (+871 ms)Consolidate snapshot: snaptb-1701143704146 at 1701143705065 (+1 ms)Loading Region manifests for snaptb-1701143704146 at 1701143705065Writing data manifest for snaptb-1701143704146 at 1701143705085 (+20 ms)Verifying snapshot: snaptb-1701143704146 at 1701143705492 (+407 ms)Snapshot snaptb-1701143704146 of table testCompactionWithHFileLink completed at 1701143705528 (+36 ms) 2023-11-28 03:55:05,530 DEBUG [PEWorker-3] locking.LockProcedure(242): UNLOCKED pid=87, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=SHARED 2023-11-28 03:55:05,532 INFO [PEWorker-3] procedure2.ProcedureExecutor(1409): Finished pid=87, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=SHARED in 1.3360 sec 2023-11-28 03:55:06,309 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-11-28 03:55:06,310 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(1212): Checking to see if snapshot from request:{ ss=snaptb-1701143704146 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-11-28 03:55:06,310 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] snapshot.SnapshotManager(403): Snapshot '{ ss=snaptb-1701143704146 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' has completed, notifying client. 2023-11-28 03:55:06,342 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-28 03:55:06,344 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] procedure.ProcedureCoordinator(143): Procedure testCompactionWithHFileLink was in running list but was completed. Accepting new attempt. 2023-11-28 03:55:06,344 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] procedure.ProcedureCoordinator(165): Submitting procedure testCompactionWithHFileLink 2023-11-28 03:55:06,344 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testCompactionWithHFileLink' 2023-11-28 03:55:06,344 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:55:06,344 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testCompactionWithHFileLink' starting 'acquire' 2023-11-28 03:55:06,344 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testCompactionWithHFileLink', kicking off acquire phase on members. 2023-11-28 03:55:06,345 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-28 03:55:06,345 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-11-28 03:55:06,347 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:55:06,347 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:06,347 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:55:06,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:55:06,347 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:06,347 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-28 03:55:06,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-11-28 03:55:06,348 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-28 03:55:06,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-28 03:55:06,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-11-28 03:55:06,348 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testCompactionWithHFileLink 2023-11-28 03:55:06,348 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testCompactionWithHFileLink is still present, removing 2023-11-28 03:55:06,349 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testCompactionWithHFileLink 2023-11-28 03:55:06,349 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testCompactionWithHFileLink' with timeout 60000ms 2023-11-28 03:55:06,349 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:55:06,349 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testCompactionWithHFileLink' starting 'acquire' stage 2023-11-28 03:55:06,350 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,,1701143695226.3d7f49de72f2e9e741f04714c6a7648c. 2023-11-28 03:55:06,350 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,,1701143695226.3d7f49de72f2e9e741f04714c6a7648c. started... 2023-11-28 03:55:06,350 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,1,1701143695226.b3c2fdb68f55a3cbe8d29ee7b6ddd5e5. 2023-11-28 03:55:06,351 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 3d7f49de72f2e9e741f04714c6a7648c 2/2 column families, dataSize=138 B heapSize=944 B 2023-11-28 03:55:06,351 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-11-28 03:55:06,351 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,2,1701143695226.2aad05dc9c73dcbea66259e5c0a6958f. 2023-11-28 03:55:06,351 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,1,1701143695226.b3c2fdb68f55a3cbe8d29ee7b6ddd5e5. started... 2023-11-28 03:55:06,351 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,2,1701143695226.2aad05dc9c73dcbea66259e5c0a6958f. started... 2023-11-28 03:55:06,351 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-28 03:55:06,351 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing b3c2fdb68f55a3cbe8d29ee7b6ddd5e5 2/2 column families, dataSize=138 B heapSize=944 B 2023-11-28 03:55:06,351 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 2aad05dc9c73dcbea66259e5c0a6958f 2/2 column families, dataSize=138 B heapSize=944 B 2023-11-28 03:55:06,778 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/.tmp/family1/d5a0c77aab9c4efc8800e1ca99af8ed0 2023-11-28 03:55:06,782 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d5a0c77aab9c4efc8800e1ca99af8ed0 2023-11-28 03:55:07,174 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/.tmp/family1/89c326765874477f9987bc51d7777ec8 2023-11-28 03:55:07,178 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 89c326765874477f9987bc51d7777ec8 2023-11-28 03:55:07,179 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/.tmp/family1/a2ac5ee01d3946638a9032263fff04ad 2023-11-28 03:55:07,183 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a2ac5ee01d3946638a9032263fff04ad 2023-11-28 03:55:07,193 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/.tmp/family2/0fbacff4fd724e8cb5071d5f6e3dd4f7 2023-11-28 03:55:07,197 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0fbacff4fd724e8cb5071d5f6e3dd4f7 2023-11-28 03:55:07,198 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/.tmp/family1/d5a0c77aab9c4efc8800e1ca99af8ed0 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family1/d5a0c77aab9c4efc8800e1ca99af8ed0 2023-11-28 03:55:07,202 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d5a0c77aab9c4efc8800e1ca99af8ed0 2023-11-28 03:55:07,202 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family1/d5a0c77aab9c4efc8800e1ca99af8ed0, entries=3, sequenceid=24, filesize=5.0 K 2023-11-28 03:55:07,202 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/.tmp/family2/0fbacff4fd724e8cb5071d5f6e3dd4f7 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family2/0fbacff4fd724e8cb5071d5f6e3dd4f7 2023-11-28 03:55:07,210 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0fbacff4fd724e8cb5071d5f6e3dd4f7 2023-11-28 03:55:07,210 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family2/0fbacff4fd724e8cb5071d5f6e3dd4f7, entries=1, sequenceid=24, filesize=5.0 K 2023-11-28 03:55:07,210 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for b3c2fdb68f55a3cbe8d29ee7b6ddd5e5 in 859ms, sequenceid=24, compaction requested=true 2023-11-28 03:55:07,210 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for b3c2fdb68f55a3cbe8d29ee7b6ddd5e5: 2023-11-28 03:55:07,211 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,1,1701143695226.b3c2fdb68f55a3cbe8d29ee7b6ddd5e5. 2023-11-28 03:55:07,211 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-11-28 03:55:07,992 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/.tmp/family2/88901938c5834359a6d4763311b40e41 2023-11-28 03:55:07,996 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 88901938c5834359a6d4763311b40e41 2023-11-28 03:55:07,996 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/.tmp/family1/89c326765874477f9987bc51d7777ec8 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family1/89c326765874477f9987bc51d7777ec8 2023-11-28 03:55:07,997 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/.tmp/family2/03ffc33ffe884981991d1f43bd43f528 2023-11-28 03:55:08,000 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 03ffc33ffe884981991d1f43bd43f528 2023-11-28 03:55:08,001 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/.tmp/family1/a2ac5ee01d3946638a9032263fff04ad as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family1/a2ac5ee01d3946638a9032263fff04ad 2023-11-28 03:55:08,001 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 89c326765874477f9987bc51d7777ec8 2023-11-28 03:55:08,001 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family1/89c326765874477f9987bc51d7777ec8, entries=3, sequenceid=24, filesize=5.0 K 2023-11-28 03:55:08,001 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/.tmp/family2/88901938c5834359a6d4763311b40e41 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family2/88901938c5834359a6d4763311b40e41 2023-11-28 03:55:08,005 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a2ac5ee01d3946638a9032263fff04ad 2023-11-28 03:55:08,005 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family1/a2ac5ee01d3946638a9032263fff04ad, entries=3, sequenceid=24, filesize=5.0 K 2023-11-28 03:55:08,005 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 88901938c5834359a6d4763311b40e41 2023-11-28 03:55:08,005 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family2/88901938c5834359a6d4763311b40e41, entries=1, sequenceid=24, filesize=5.0 K 2023-11-28 03:55:08,005 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/.tmp/family2/03ffc33ffe884981991d1f43bd43f528 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family2/03ffc33ffe884981991d1f43bd43f528 2023-11-28 03:55:08,006 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 2aad05dc9c73dcbea66259e5c0a6958f in 1655ms, sequenceid=24, compaction requested=true 2023-11-28 03:55:08,006 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 2aad05dc9c73dcbea66259e5c0a6958f: 2023-11-28 03:55:08,006 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,2,1701143695226.2aad05dc9c73dcbea66259e5c0a6958f. 2023-11-28 03:55:08,006 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-11-28 03:55:08,009 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 03ffc33ffe884981991d1f43bd43f528 2023-11-28 03:55:08,009 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family2/03ffc33ffe884981991d1f43bd43f528, entries=1, sequenceid=24, filesize=5.0 K 2023-11-28 03:55:08,010 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 3d7f49de72f2e9e741f04714c6a7648c in 1659ms, sequenceid=24, compaction requested=true 2023-11-28 03:55:08,010 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 3d7f49de72f2e9e741f04714c6a7648c: 2023-11-28 03:55:08,010 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,,1701143695226.3d7f49de72f2e9e741f04714c6a7648c. 2023-11-28 03:55:08,010 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-11-28 03:55:08,010 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-11-28 03:55:08,010 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:08,010 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testCompactionWithHFileLink' locally acquired 2023-11-28 03:55:08,010 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure (testCompactionWithHFileLink) in zk 2023-11-28 03:55:08,012 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-28 03:55:08,012 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:08,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:08,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:55:08,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:55:08,012 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-28 03:55:08,012 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testCompactionWithHFileLink' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-28 03:55:08,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:55:08,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:55:08,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-11-28 03:55:08,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:08,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:55:08,013 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure 'testCompactionWithHFileLink' on coordinator 2023-11-28 03:55:08,013 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2c64698e[Count = 0] remaining members to acquire global barrier 2023-11-28 03:55:08,013 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testCompactionWithHFileLink' starting 'in-barrier' execution. 2023-11-28 03:55:08,013 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-28 03:55:08,015 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-28 03:55:08,015 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-28 03:55:08,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-28 03:55:08,015 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testCompactionWithHFileLink' received 'reached' from coordinator. 2023-11-28 03:55:08,015 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testCompactionWithHFileLink' locally completed 2023-11-28 03:55:08,015 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testCompactionWithHFileLink' completed for member 'jenkins-hbase4.apache.org,41203,1701143600653' in zk 2023-11-28 03:55:08,015 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:08,015 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-28 03:55:08,016 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testCompactionWithHFileLink' has notified controller of completion 2023-11-28 03:55:08,016 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:08,016 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:55:08,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:08,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:55:08,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:55:08,016 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testCompactionWithHFileLink' completed. 2023-11-28 03:55:08,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:55:08,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:55:08,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-11-28 03:55:08,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:08,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:55:08,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-11-28 03:55:08,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:08,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testCompactionWithHFileLink' member 'jenkins-hbase4.apache.org,41203,1701143600653': 2023-11-28 03:55:08,018 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' released barrier for procedure'testCompactionWithHFileLink', counting down latch. Waiting for 0 more 2023-11-28 03:55:08,018 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testCompactionWithHFileLink' execution completed 2023-11-28 03:55:08,018 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-28 03:55:08,019 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-28 03:55:08,019 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testCompactionWithHFileLink 2023-11-28 03:55:08,019 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testCompactionWithHFileLinkincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-28 03:55:08,021 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-28 03:55:08,021 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-28 03:55:08,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-28 03:55:08,021 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-28 03:55:08,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-28 03:55:08,021 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:55:08,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:55:08,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:55:08,022 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:55:08,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:55:08,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:55:08,022 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:08,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-28 03:55:08,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-11-28 03:55:08,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:55:08,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-11-28 03:55:08,023 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:08,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:08,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:55:08,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-11-28 03:55:08,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:08,025 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:08,025 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:55:08,026 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-11-28 03:55:08,026 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-11-28 03:55:08,026 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testCompactionWithHFileLink' 2023-11-28 03:55:08,026 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(399): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink because node does not exist (not an error) 2023-11-28 03:55:08,026 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:55:08,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:55:08,026 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:55:08,026 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-28 03:55:08,026 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:08,026 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:55:08,026 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-28 03:55:08,026 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-28 03:55:08,026 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-28 03:55:08,026 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:55:08,026 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testCompactionWithHFileLink'' to complete. (max 300000 ms per retry) 2023-11-28 03:55:08,026 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-28 03:55:08,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:55:08,126 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-28 03:55:08,127 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38593] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-28 03:55:08,128 DEBUG [Listener at localhost/42853] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-11-28 03:55:08,128 DEBUG [Listener at localhost/42853] regionserver.HStore(1912): 3d7f49de72f2e9e741f04714c6a7648c/family1 is initiating major compaction (all files) 2023-11-28 03:55:08,128 INFO [Listener at localhost/42853] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-11-28 03:55:08,128 INFO [Listener at localhost/42853] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-11-28 03:55:08,129 INFO [Listener at localhost/42853] regionserver.HRegion(2259): Starting compaction of 3d7f49de72f2e9e741f04714c6a7648c/family1 in testCompactionWithHFileLink,,1701143695226.3d7f49de72f2e9e741f04714c6a7648c. 2023-11-28 03:55:08,129 INFO [Listener at localhost/42853] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family1/9cba99270b974d6b9181045f6f7ee6bc, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family1/149c75fc3d8a444bbd7d2180c2614583, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family1/b313b9e7ed474d04bf592f578c1928e8, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family1/0f49b13a3dc94f1bb9ed692753ce2e94, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family1/a2ac5ee01d3946638a9032263fff04ad] into tmpdir=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/.tmp, totalSize=26.7 K 2023-11-28 03:55:08,129 DEBUG [Listener at localhost/42853] compactions.Compactor(207): Compacting 9cba99270b974d6b9181045f6f7ee6bc, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1701143697342 2023-11-28 03:55:08,130 DEBUG [Listener at localhost/42853] compactions.Compactor(207): Compacting 149c75fc3d8a444bbd7d2180c2614583, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1701143698738 2023-11-28 03:55:08,130 DEBUG [Listener at localhost/42853] compactions.Compactor(207): Compacting b313b9e7ed474d04bf592f578c1928e8, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1701143700547 2023-11-28 03:55:08,130 DEBUG [Listener at localhost/42853] compactions.Compactor(207): Compacting 0f49b13a3dc94f1bb9ed692753ce2e94, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1701143702350 2023-11-28 03:55:08,131 DEBUG [Listener at localhost/42853] compactions.Compactor(207): Compacting a2ac5ee01d3946638a9032263fff04ad, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-11-28 03:55:08,143 INFO [Listener at localhost/42853] throttle.PressureAwareThroughputController(145): 3d7f49de72f2e9e741f04714c6a7648c#family1#compaction#107 average throughput is 0.57 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-11-28 03:55:08,553 INFO [Listener at localhost/42853] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e20231128e3e9d5da083145f088ada4b38be067fe_del 2023-11-28 03:55:08,553 INFO [Listener at localhost/42853] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20231128e3e9d5da083145f088ada4b38be067fe_del to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20231128e3e9d5da083145f088ada4b38be067fe_del 2023-11-28 03:55:09,368 INFO [Listener at localhost/42853] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2c5387a00c534c1ba4b9b3b4a82a16db 2023-11-28 03:55:09,369 DEBUG [Listener at localhost/42853] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/.tmp/family1/2c5387a00c534c1ba4b9b3b4a82a16db as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family1/2c5387a00c534c1ba4b9b3b4a82a16db 2023-11-28 03:55:09,373 INFO [Listener at localhost/42853] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2c5387a00c534c1ba4b9b3b4a82a16db 2023-11-28 03:55:09,374 INFO [Listener at localhost/42853] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 3d7f49de72f2e9e741f04714c6a7648c/family1 of 3d7f49de72f2e9e741f04714c6a7648c into 2c5387a00c534c1ba4b9b3b4a82a16db(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-11-28 03:55:09,374 DEBUG [Listener at localhost/42853] regionserver.HRegion(2289): Compaction status journal for 3d7f49de72f2e9e741f04714c6a7648c: 2023-11-28 03:55:09,374 DEBUG [Listener at localhost/42853] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-11-28 03:55:09,374 DEBUG [Listener at localhost/42853] regionserver.HStore(1912): 3d7f49de72f2e9e741f04714c6a7648c/family2 is initiating major compaction (all files) 2023-11-28 03:55:09,374 INFO [Listener at localhost/42853] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-11-28 03:55:09,374 INFO [Listener at localhost/42853] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-11-28 03:55:09,374 INFO [Listener at localhost/42853] regionserver.HRegion(2259): Starting compaction of 3d7f49de72f2e9e741f04714c6a7648c/family2 in testCompactionWithHFileLink,,1701143695226.3d7f49de72f2e9e741f04714c6a7648c. 2023-11-28 03:55:09,374 INFO [Listener at localhost/42853] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family2/cfbbdda0c47f440ab7343ce95b4baaf5, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family2/14783dff7d074f04ac3290bc0e4a92ba, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family2/cf1816b4243e4ad197a1721e4235af5a, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family2/a7b3c563e013482e86b9f3270fa9799c, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family2/03ffc33ffe884981991d1f43bd43f528] into tmpdir=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/.tmp, totalSize=25.4 K 2023-11-28 03:55:09,375 DEBUG [Listener at localhost/42853] compactions.Compactor(207): Compacting cfbbdda0c47f440ab7343ce95b4baaf5, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1701143697342 2023-11-28 03:55:09,375 DEBUG [Listener at localhost/42853] compactions.Compactor(207): Compacting 14783dff7d074f04ac3290bc0e4a92ba, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1701143698738 2023-11-28 03:55:09,375 DEBUG [Listener at localhost/42853] compactions.Compactor(207): Compacting cf1816b4243e4ad197a1721e4235af5a, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1701143700547 2023-11-28 03:55:09,376 DEBUG [Listener at localhost/42853] compactions.Compactor(207): Compacting a7b3c563e013482e86b9f3270fa9799c, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1701143702350 2023-11-28 03:55:09,376 DEBUG [Listener at localhost/42853] compactions.Compactor(207): Compacting 03ffc33ffe884981991d1f43bd43f528, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-11-28 03:55:09,387 INFO [Listener at localhost/42853] throttle.PressureAwareThroughputController(145): 3d7f49de72f2e9e741f04714c6a7648c#family2#compaction#108 average throughput is 0.35 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-11-28 03:55:09,407 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family1/9cba99270b974d6b9181045f6f7ee6bc, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family1/149c75fc3d8a444bbd7d2180c2614583, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family1/b313b9e7ed474d04bf592f578c1928e8, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family1/0f49b13a3dc94f1bb9ed692753ce2e94, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family1/a2ac5ee01d3946638a9032263fff04ad] to archive 2023-11-28 03:55:09,408 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-28 03:55:09,409 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family1/9cba99270b974d6b9181045f6f7ee6bc to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family1/9cba99270b974d6b9181045f6f7ee6bc 2023-11-28 03:55:09,410 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family1/149c75fc3d8a444bbd7d2180c2614583 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family1/149c75fc3d8a444bbd7d2180c2614583 2023-11-28 03:55:09,411 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family1/b313b9e7ed474d04bf592f578c1928e8 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family1/b313b9e7ed474d04bf592f578c1928e8 2023-11-28 03:55:09,412 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family1/0f49b13a3dc94f1bb9ed692753ce2e94 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family1/0f49b13a3dc94f1bb9ed692753ce2e94 2023-11-28 03:55:09,413 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family1/a2ac5ee01d3946638a9032263fff04ad to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family1/a2ac5ee01d3946638a9032263fff04ad 2023-11-28 03:55:09,796 INFO [Listener at localhost/42853] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e20231128203e460f7da34a628c95173a08d56f3b_del 2023-11-28 03:55:09,797 INFO [Listener at localhost/42853] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20231128203e460f7da34a628c95173a08d56f3b_del to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20231128203e460f7da34a628c95173a08d56f3b_del 2023-11-28 03:55:10,610 INFO [Listener at localhost/42853] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 79ff143814eb45a58e52ea79560b8f93 2023-11-28 03:55:10,611 DEBUG [Listener at localhost/42853] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/.tmp/family2/79ff143814eb45a58e52ea79560b8f93 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family2/79ff143814eb45a58e52ea79560b8f93 2023-11-28 03:55:10,614 INFO [Listener at localhost/42853] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 79ff143814eb45a58e52ea79560b8f93 2023-11-28 03:55:10,615 INFO [Listener at localhost/42853] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 3d7f49de72f2e9e741f04714c6a7648c/family2 of 3d7f49de72f2e9e741f04714c6a7648c into 79ff143814eb45a58e52ea79560b8f93(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-11-28 03:55:10,615 DEBUG [Listener at localhost/42853] regionserver.HRegion(2289): Compaction status journal for 3d7f49de72f2e9e741f04714c6a7648c: 2023-11-28 03:55:10,615 DEBUG [Listener at localhost/42853] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-11-28 03:55:10,615 DEBUG [Listener at localhost/42853] regionserver.HStore(1912): b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family1 is initiating major compaction (all files) 2023-11-28 03:55:10,615 INFO [Listener at localhost/42853] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-11-28 03:55:10,615 INFO [Listener at localhost/42853] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-11-28 03:55:10,616 INFO [Listener at localhost/42853] regionserver.HRegion(2259): Starting compaction of b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family1 in testCompactionWithHFileLink,1,1701143695226.b3c2fdb68f55a3cbe8d29ee7b6ddd5e5. 2023-11-28 03:55:10,616 INFO [Listener at localhost/42853] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family1/88a76a3cbca146b6bb71214ecc4a9c87, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family1/299d98e37656434ea5b2482137c58710, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family1/dc7b41ef849648dda6e80bfb0cdeea0f, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family1/34bdbacde3fd4331ae0cc21041140fd9, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family1/d5a0c77aab9c4efc8800e1ca99af8ed0] into tmpdir=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/.tmp, totalSize=26.7 K 2023-11-28 03:55:10,616 DEBUG [Listener at localhost/42853] compactions.Compactor(207): Compacting 88a76a3cbca146b6bb71214ecc4a9c87, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1701143697344 2023-11-28 03:55:10,617 DEBUG [Listener at localhost/42853] compactions.Compactor(207): Compacting 299d98e37656434ea5b2482137c58710, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1701143698739 2023-11-28 03:55:10,617 DEBUG [Listener at localhost/42853] compactions.Compactor(207): Compacting dc7b41ef849648dda6e80bfb0cdeea0f, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1701143700547 2023-11-28 03:55:10,617 DEBUG [Listener at localhost/42853] compactions.Compactor(207): Compacting 34bdbacde3fd4331ae0cc21041140fd9, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1701143702350 2023-11-28 03:55:10,618 DEBUG [Listener at localhost/42853] compactions.Compactor(207): Compacting d5a0c77aab9c4efc8800e1ca99af8ed0, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-11-28 03:55:10,627 INFO [Listener at localhost/42853] throttle.PressureAwareThroughputController(145): b3c2fdb68f55a3cbe8d29ee7b6ddd5e5#family1#compaction#109 average throughput is 0.85 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-11-28 03:55:10,711 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family2/cfbbdda0c47f440ab7343ce95b4baaf5, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family2/14783dff7d074f04ac3290bc0e4a92ba, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family2/cf1816b4243e4ad197a1721e4235af5a, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family2/a7b3c563e013482e86b9f3270fa9799c, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family2/03ffc33ffe884981991d1f43bd43f528] to archive 2023-11-28 03:55:10,712 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-28 03:55:10,714 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family2/cfbbdda0c47f440ab7343ce95b4baaf5 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family2/cfbbdda0c47f440ab7343ce95b4baaf5 2023-11-28 03:55:10,715 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family2/14783dff7d074f04ac3290bc0e4a92ba to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family2/14783dff7d074f04ac3290bc0e4a92ba 2023-11-28 03:55:10,716 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family2/cf1816b4243e4ad197a1721e4235af5a to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family2/cf1816b4243e4ad197a1721e4235af5a 2023-11-28 03:55:10,717 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family2/a7b3c563e013482e86b9f3270fa9799c to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family2/a7b3c563e013482e86b9f3270fa9799c 2023-11-28 03:55:10,719 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family2/03ffc33ffe884981991d1f43bd43f528 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family2/03ffc33ffe884981991d1f43bd43f528 2023-11-28 03:55:11,035 INFO [Listener at localhost/42853] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b2023112804c33ea8ae7e4de4aec70160568ac534_del 2023-11-28 03:55:11,036 INFO [Listener at localhost/42853] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023112804c33ea8ae7e4de4aec70160568ac534_del to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023112804c33ea8ae7e4de4aec70160568ac534_del 2023-11-28 03:55:11,850 INFO [Listener at localhost/42853] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2c88d564dad043448d1c89f73401463e 2023-11-28 03:55:11,851 DEBUG [Listener at localhost/42853] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/.tmp/family1/2c88d564dad043448d1c89f73401463e as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family1/2c88d564dad043448d1c89f73401463e 2023-11-28 03:55:11,854 INFO [Listener at localhost/42853] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2c88d564dad043448d1c89f73401463e 2023-11-28 03:55:11,855 INFO [Listener at localhost/42853] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family1 of b3c2fdb68f55a3cbe8d29ee7b6ddd5e5 into 2c88d564dad043448d1c89f73401463e(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-11-28 03:55:11,855 DEBUG [Listener at localhost/42853] regionserver.HRegion(2289): Compaction status journal for b3c2fdb68f55a3cbe8d29ee7b6ddd5e5: 2023-11-28 03:55:11,855 DEBUG [Listener at localhost/42853] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-11-28 03:55:11,855 DEBUG [Listener at localhost/42853] regionserver.HStore(1912): b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family2 is initiating major compaction (all files) 2023-11-28 03:55:11,855 INFO [Listener at localhost/42853] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-11-28 03:55:11,855 INFO [Listener at localhost/42853] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-11-28 03:55:11,856 INFO [Listener at localhost/42853] regionserver.HRegion(2259): Starting compaction of b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family2 in testCompactionWithHFileLink,1,1701143695226.b3c2fdb68f55a3cbe8d29ee7b6ddd5e5. 2023-11-28 03:55:11,856 INFO [Listener at localhost/42853] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family2/ee2d50b684b2479e8a02236cf5083faa, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family2/75ff1f4a28e9466e86a2f50fe4b3ba7c, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family2/b06633c919b549ed9628414e3a0d7c29, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family2/fe11679925e74afa9cd10ab81215b48a, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family2/0fbacff4fd724e8cb5071d5f6e3dd4f7] into tmpdir=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/.tmp, totalSize=25.4 K 2023-11-28 03:55:11,856 DEBUG [Listener at localhost/42853] compactions.Compactor(207): Compacting ee2d50b684b2479e8a02236cf5083faa, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1701143697344 2023-11-28 03:55:11,856 DEBUG [Listener at localhost/42853] compactions.Compactor(207): Compacting 75ff1f4a28e9466e86a2f50fe4b3ba7c, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1701143698739 2023-11-28 03:55:11,857 DEBUG [Listener at localhost/42853] compactions.Compactor(207): Compacting b06633c919b549ed9628414e3a0d7c29, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1701143700547 2023-11-28 03:55:11,857 DEBUG [Listener at localhost/42853] compactions.Compactor(207): Compacting fe11679925e74afa9cd10ab81215b48a, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1701143702350 2023-11-28 03:55:11,857 DEBUG [Listener at localhost/42853] compactions.Compactor(207): Compacting 0fbacff4fd724e8cb5071d5f6e3dd4f7, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-11-28 03:55:11,869 INFO [Listener at localhost/42853] throttle.PressureAwareThroughputController(145): b3c2fdb68f55a3cbe8d29ee7b6ddd5e5#family2#compaction#110 average throughput is 0.35 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-11-28 03:55:11,905 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family1/88a76a3cbca146b6bb71214ecc4a9c87, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family1/299d98e37656434ea5b2482137c58710, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family1/dc7b41ef849648dda6e80bfb0cdeea0f, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family1/34bdbacde3fd4331ae0cc21041140fd9, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family1/d5a0c77aab9c4efc8800e1ca99af8ed0] to archive 2023-11-28 03:55:11,906 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-28 03:55:11,907 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family1/88a76a3cbca146b6bb71214ecc4a9c87 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family1/88a76a3cbca146b6bb71214ecc4a9c87 2023-11-28 03:55:11,908 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family1/299d98e37656434ea5b2482137c58710 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family1/299d98e37656434ea5b2482137c58710 2023-11-28 03:55:11,909 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family1/dc7b41ef849648dda6e80bfb0cdeea0f to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family1/dc7b41ef849648dda6e80bfb0cdeea0f 2023-11-28 03:55:11,910 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family1/34bdbacde3fd4331ae0cc21041140fd9 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family1/34bdbacde3fd4331ae0cc21041140fd9 2023-11-28 03:55:11,911 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family1/d5a0c77aab9c4efc8800e1ca99af8ed0 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family1/d5a0c77aab9c4efc8800e1ca99af8ed0 2023-11-28 03:55:12,276 INFO [Listener at localhost/42853] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b2023112847e6b2a73e5648f988213810b241c45a_del 2023-11-28 03:55:12,276 INFO [Listener at localhost/42853] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023112847e6b2a73e5648f988213810b241c45a_del to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b2023112847e6b2a73e5648f988213810b241c45a_del 2023-11-28 03:55:13,091 INFO [Listener at localhost/42853] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4c67664540a48089e64216c5442c7b5 2023-11-28 03:55:13,091 DEBUG [Listener at localhost/42853] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/.tmp/family2/c4c67664540a48089e64216c5442c7b5 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family2/c4c67664540a48089e64216c5442c7b5 2023-11-28 03:55:13,095 INFO [Listener at localhost/42853] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4c67664540a48089e64216c5442c7b5 2023-11-28 03:55:13,096 INFO [Listener at localhost/42853] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family2 of b3c2fdb68f55a3cbe8d29ee7b6ddd5e5 into c4c67664540a48089e64216c5442c7b5(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-11-28 03:55:13,096 DEBUG [Listener at localhost/42853] regionserver.HRegion(2289): Compaction status journal for b3c2fdb68f55a3cbe8d29ee7b6ddd5e5: 2023-11-28 03:55:13,096 DEBUG [Listener at localhost/42853] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-11-28 03:55:13,096 DEBUG [Listener at localhost/42853] regionserver.HStore(1912): 2aad05dc9c73dcbea66259e5c0a6958f/family1 is initiating major compaction (all files) 2023-11-28 03:55:13,096 INFO [Listener at localhost/42853] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-11-28 03:55:13,096 INFO [Listener at localhost/42853] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-11-28 03:55:13,096 INFO [Listener at localhost/42853] regionserver.HRegion(2259): Starting compaction of 2aad05dc9c73dcbea66259e5c0a6958f/family1 in testCompactionWithHFileLink,2,1701143695226.2aad05dc9c73dcbea66259e5c0a6958f. 2023-11-28 03:55:13,096 INFO [Listener at localhost/42853] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family1/5bd4594355ca45ff98e17e8e298757d3, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family1/ef8ca9f746b4468d8da6b892ab104927, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family1/55b0944f49f248fd87ca616d0ed03e14, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family1/ab034fef3fd24871aeeb0dc81dfe66c2, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family1/89c326765874477f9987bc51d7777ec8] into tmpdir=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/.tmp, totalSize=26.7 K 2023-11-28 03:55:13,097 DEBUG [Listener at localhost/42853] compactions.Compactor(207): Compacting 5bd4594355ca45ff98e17e8e298757d3, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1701143697344 2023-11-28 03:55:13,097 DEBUG [Listener at localhost/42853] compactions.Compactor(207): Compacting ef8ca9f746b4468d8da6b892ab104927, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1701143698739 2023-11-28 03:55:13,097 DEBUG [Listener at localhost/42853] compactions.Compactor(207): Compacting 55b0944f49f248fd87ca616d0ed03e14, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1701143700547 2023-11-28 03:55:13,098 DEBUG [Listener at localhost/42853] compactions.Compactor(207): Compacting ab034fef3fd24871aeeb0dc81dfe66c2, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1701143702350 2023-11-28 03:55:13,098 DEBUG [Listener at localhost/42853] compactions.Compactor(207): Compacting 89c326765874477f9987bc51d7777ec8, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-11-28 03:55:13,105 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family2/ee2d50b684b2479e8a02236cf5083faa, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family2/75ff1f4a28e9466e86a2f50fe4b3ba7c, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family2/b06633c919b549ed9628414e3a0d7c29, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family2/fe11679925e74afa9cd10ab81215b48a, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family2/0fbacff4fd724e8cb5071d5f6e3dd4f7] to archive 2023-11-28 03:55:13,106 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-28 03:55:13,107 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family2/ee2d50b684b2479e8a02236cf5083faa to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family2/ee2d50b684b2479e8a02236cf5083faa 2023-11-28 03:55:13,108 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family2/75ff1f4a28e9466e86a2f50fe4b3ba7c to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family2/75ff1f4a28e9466e86a2f50fe4b3ba7c 2023-11-28 03:55:13,108 INFO [Listener at localhost/42853] throttle.PressureAwareThroughputController(145): 2aad05dc9c73dcbea66259e5c0a6958f#family1#compaction#111 average throughput is 0.85 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-11-28 03:55:13,109 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family2/b06633c919b549ed9628414e3a0d7c29 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family2/b06633c919b549ed9628414e3a0d7c29 2023-11-28 03:55:13,110 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family2/fe11679925e74afa9cd10ab81215b48a to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family2/fe11679925e74afa9cd10ab81215b48a 2023-11-28 03:55:13,112 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family2/0fbacff4fd724e8cb5071d5f6e3dd4f7 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family2/0fbacff4fd724e8cb5071d5f6e3dd4f7 2023-11-28 03:55:13,516 INFO [Listener at localhost/42853] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c2023112884b65916c9774d4c8416f134fef2f12f_del 2023-11-28 03:55:13,517 INFO [Listener at localhost/42853] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023112884b65916c9774d4c8416f134fef2f12f_del to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023112884b65916c9774d4c8416f134fef2f12f_del 2023-11-28 03:55:14,333 INFO [Listener at localhost/42853] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for b54918ba975e4d51b6b332b5e3e581de 2023-11-28 03:55:14,334 DEBUG [Listener at localhost/42853] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/.tmp/family1/b54918ba975e4d51b6b332b5e3e581de as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family1/b54918ba975e4d51b6b332b5e3e581de 2023-11-28 03:55:14,337 INFO [Listener at localhost/42853] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for b54918ba975e4d51b6b332b5e3e581de 2023-11-28 03:55:14,338 INFO [Listener at localhost/42853] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 2aad05dc9c73dcbea66259e5c0a6958f/family1 of 2aad05dc9c73dcbea66259e5c0a6958f into b54918ba975e4d51b6b332b5e3e581de(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-11-28 03:55:14,338 DEBUG [Listener at localhost/42853] regionserver.HRegion(2289): Compaction status journal for 2aad05dc9c73dcbea66259e5c0a6958f: 2023-11-28 03:55:14,339 DEBUG [Listener at localhost/42853] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-11-28 03:55:14,339 DEBUG [Listener at localhost/42853] regionserver.HStore(1912): 2aad05dc9c73dcbea66259e5c0a6958f/family2 is initiating major compaction (all files) 2023-11-28 03:55:14,339 INFO [Listener at localhost/42853] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-11-28 03:55:14,339 INFO [Listener at localhost/42853] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-11-28 03:55:14,339 INFO [Listener at localhost/42853] regionserver.HRegion(2259): Starting compaction of 2aad05dc9c73dcbea66259e5c0a6958f/family2 in testCompactionWithHFileLink,2,1701143695226.2aad05dc9c73dcbea66259e5c0a6958f. 2023-11-28 03:55:14,339 INFO [Listener at localhost/42853] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family2/a79a53b7062d4e53b9199b02c23f6c24, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family2/a5d2b1b5f59d4745beda10e8ec8e729f, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family2/9d803becce0f4ac4ad8171434ff9b0a2, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family2/cdc6a50b83e147878684e9d41c01c3e6, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family2/88901938c5834359a6d4763311b40e41] into tmpdir=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/.tmp, totalSize=25.4 K 2023-11-28 03:55:14,339 DEBUG [Listener at localhost/42853] compactions.Compactor(207): Compacting a79a53b7062d4e53b9199b02c23f6c24, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1701143697344 2023-11-28 03:55:14,340 DEBUG [Listener at localhost/42853] compactions.Compactor(207): Compacting a5d2b1b5f59d4745beda10e8ec8e729f, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1701143698739 2023-11-28 03:55:14,340 DEBUG [Listener at localhost/42853] compactions.Compactor(207): Compacting 9d803becce0f4ac4ad8171434ff9b0a2, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1701143700547 2023-11-28 03:55:14,340 DEBUG [Listener at localhost/42853] compactions.Compactor(207): Compacting cdc6a50b83e147878684e9d41c01c3e6, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1701143702350 2023-11-28 03:55:14,341 DEBUG [Listener at localhost/42853] compactions.Compactor(207): Compacting 88901938c5834359a6d4763311b40e41, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-11-28 03:55:14,352 INFO [Listener at localhost/42853] throttle.PressureAwareThroughputController(145): 2aad05dc9c73dcbea66259e5c0a6958f#family2#compaction#112 average throughput is 0.53 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-11-28 03:55:14,405 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family1/5bd4594355ca45ff98e17e8e298757d3, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family1/ef8ca9f746b4468d8da6b892ab104927, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family1/55b0944f49f248fd87ca616d0ed03e14, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family1/ab034fef3fd24871aeeb0dc81dfe66c2, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family1/89c326765874477f9987bc51d7777ec8] to archive 2023-11-28 03:55:14,406 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-28 03:55:14,408 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family1/5bd4594355ca45ff98e17e8e298757d3 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family1/5bd4594355ca45ff98e17e8e298757d3 2023-11-28 03:55:14,409 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family1/ef8ca9f746b4468d8da6b892ab104927 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family1/ef8ca9f746b4468d8da6b892ab104927 2023-11-28 03:55:14,410 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family1/55b0944f49f248fd87ca616d0ed03e14 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family1/55b0944f49f248fd87ca616d0ed03e14 2023-11-28 03:55:14,411 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family1/ab034fef3fd24871aeeb0dc81dfe66c2 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family1/ab034fef3fd24871aeeb0dc81dfe66c2 2023-11-28 03:55:14,412 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family1/89c326765874477f9987bc51d7777ec8 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family1/89c326765874477f9987bc51d7777ec8 2023-11-28 03:55:14,760 INFO [Listener at localhost/42853] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c20231128f2a1b19354fb4a1ea87f959529de2a75_del 2023-11-28 03:55:14,760 INFO [Listener at localhost/42853] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20231128f2a1b19354fb4a1ea87f959529de2a75_del to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20231128f2a1b19354fb4a1ea87f959529de2a75_del 2023-11-28 03:55:15,575 INFO [Listener at localhost/42853] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for b93f7d6b9dbf4058937d8aa40eb41bb5 2023-11-28 03:55:15,576 DEBUG [Listener at localhost/42853] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/.tmp/family2/b93f7d6b9dbf4058937d8aa40eb41bb5 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family2/b93f7d6b9dbf4058937d8aa40eb41bb5 2023-11-28 03:55:15,579 INFO [Listener at localhost/42853] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for b93f7d6b9dbf4058937d8aa40eb41bb5 2023-11-28 03:55:15,580 INFO [Listener at localhost/42853] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 2aad05dc9c73dcbea66259e5c0a6958f/family2 of 2aad05dc9c73dcbea66259e5c0a6958f into b93f7d6b9dbf4058937d8aa40eb41bb5(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-11-28 03:55:15,580 DEBUG [Listener at localhost/42853] regionserver.HRegion(2289): Compaction status journal for 2aad05dc9c73dcbea66259e5c0a6958f: 2023-11-28 03:55:15,585 DEBUG [Listener at localhost/42853] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-11-28 03:55:15,589 DEBUG [Listener at localhost/42853] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-11-28 03:55:15,594 DEBUG [Listener at localhost/42853] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-11-28 03:55:15,595 DEBUG [Listener at localhost/42853] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-11-28 03:55:15,604 INFO [Listener at localhost/42853] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-11-28 03:55:15,605 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family2/a79a53b7062d4e53b9199b02c23f6c24, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family2/a5d2b1b5f59d4745beda10e8ec8e729f, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family2/9d803becce0f4ac4ad8171434ff9b0a2, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family2/cdc6a50b83e147878684e9d41c01c3e6, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family2/88901938c5834359a6d4763311b40e41] to archive 2023-11-28 03:55:15,606 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-28 03:55:15,608 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family2/a79a53b7062d4e53b9199b02c23f6c24 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family2/a79a53b7062d4e53b9199b02c23f6c24 2023-11-28 03:55:15,610 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family2/a5d2b1b5f59d4745beda10e8ec8e729f to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family2/a5d2b1b5f59d4745beda10e8ec8e729f 2023-11-28 03:55:15,612 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family2/9d803becce0f4ac4ad8171434ff9b0a2 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family2/9d803becce0f4ac4ad8171434ff9b0a2 2023-11-28 03:55:15,613 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family2/cdc6a50b83e147878684e9d41c01c3e6 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family2/cdc6a50b83e147878684e9d41c01c3e6 2023-11-28 03:55:15,614 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family2/88901938c5834359a6d4763311b40e41 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family2/88901938c5834359a6d4763311b40e41 2023-11-28 03:55:15,638 INFO [Listener at localhost/42853] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 3 del files, 12 selected files, and 0 irrelevant files table 'testCompactionWithHFileLink' and column 'family1' 2023-11-28 03:55:15,641 INFO [Listener at localhost/42853] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e20231128e3e9d5da083145f088ada4b38be067fe_del 2023-11-28 03:55:15,644 INFO [Listener at localhost/42853] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b2023112804c33ea8ae7e4de4aec70160568ac534_del 2023-11-28 03:55:15,646 INFO [Listener at localhost/42853] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c2023112884b65916c9774d4c8416f134fef2f12f_del 2023-11-28 03:55:15,646 INFO [Listener at localhost/42853] compactions.PartitionedMobCompactor(350): After merging, there are 3 del files. table='testCompactionWithHFileLink' column='family1' 2023-11-28 03:55:15,647 DEBUG [Listener at localhost/42853] zookeeper.ReadOnlyZKClient(139): Connect 0x447597be to 127.0.0.1:55641 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-11-28 03:55:15,652 DEBUG [Listener at localhost/42853] ipc.AbstractRpcClient(189): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@77b53008, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-11-28 03:55:15,652 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20231128 for table='testCompactionWithHFileLink' and column='family1' 2023-11-28 03:55:15,652 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c81e728d9d4c2f636f067f89cc14862c20231128 for table='testCompactionWithHFileLink' and column='family1' 2023-11-28 03:55:15,653 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c4ca4238a0b923820dcc509a6f75849b20231128 for table='testCompactionWithHFileLink' and column='family1' 2023-11-28 03:55:16,522 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20231128e981aa212a084c27a0d588aba8fd58d9 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20231128e981aa212a084c27a0d588aba8fd58d9 2023-11-28 03:55:16,522 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20231128ff787fef44d04a7e9bac61acbfca183b to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20231128ff787fef44d04a7e9bac61acbfca183b 2023-11-28 03:55:16,522 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20231128d2c59f3654234b0db8ace0004962cc6d to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20231128d2c59f3654234b0db8ace0004962cc6d 2023-11-28 03:55:16,523 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20231128 table='testCompactionWithHFileLink' column='family1' 2023-11-28 03:55:16,523 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20231128 table='testCompactionWithHFileLink' column='family1' 2023-11-28 03:55:16,523 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20231128 table='testCompactionWithHFileLink' column='family1' 2023-11-28 03:55:16,537 DEBUG [MobFileCompactionChore-pool-2] ipc.RpcConnection(122): Using SIMPLE authentication for service=ClientService, sasl=false 2023-11-28 03:55:16,540 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:55312, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-11-28 03:55:16,548 DEBUG [MobFileCompactionChore-pool-2] ipc.RpcConnection(122): Using SIMPLE authentication for service=MasterService, sasl=false 2023-11-28 03:55:16,553 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:39768, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-11-28 03:55:16,568 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20231128/family1/da2ed9d2a4bb40cc94bb3d5f1565feb3 first=Optional[1\x00\x00\x00\x01] last=Optional[1\x00\x00\x00\x07] 2023-11-28 03:55:16,568 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20231128/family1/f7aa4e01c9d249ce8ea68c5354267bc0 first=Optional[2\x00\x00\x00\x01] last=Optional[2\x00\x00\x00\x07] 2023-11-28 03:55:16,569 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testCompactionWithHFileLink,1,1701143695226.b3c2fdb68f55a3cbe8d29ee7b6ddd5e5., hostname=jenkins-hbase4.apache.org,41203,1701143600653, seqNum=2 for row 1 with hfile group [{family1,hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20231128/family1/da2ed9d2a4bb40cc94bb3d5f1565feb3}] 2023-11-28 03:55:16,569 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20231128/family1/ffcdb65d86ed4f4187cb31ec0df9e2fa first=Optional[0\x00\x00\x00\x01] last=Optional[0\x00\x00\x00\x07] 2023-11-28 03:55:16,571 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testCompactionWithHFileLink,2,1701143695226.2aad05dc9c73dcbea66259e5c0a6958f., hostname=jenkins-hbase4.apache.org,41203,1701143600653, seqNum=2 for row 2 with hfile group [{family1,hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20231128/family1/f7aa4e01c9d249ce8ea68c5354267bc0}] 2023-11-28 03:55:16,571 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testCompactionWithHFileLink,,1701143695226.3d7f49de72f2e9e741f04714c6a7648c., hostname=jenkins-hbase4.apache.org,41203,1701143600653, seqNum=2 for row with hfile group [{family1,hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20231128/family1/ffcdb65d86ed4f4187cb31ec0df9e2fa}] 2023-11-28 03:55:16,587 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41203] regionserver.HStore(763): Validating hfile at hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20231128/family1/ffcdb65d86ed4f4187cb31ec0df9e2fa for inclusion in 3d7f49de72f2e9e741f04714c6a7648c/family1 2023-11-28 03:55:16,587 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=41203] regionserver.HStore(763): Validating hfile at hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20231128/family1/da2ed9d2a4bb40cc94bb3d5f1565feb3 for inclusion in b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family1 2023-11-28 03:55:16,587 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HStore(763): Validating hfile at hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20231128/family1/f7aa4e01c9d249ce8ea68c5354267bc0 for inclusion in 2aad05dc9c73dcbea66259e5c0a6958f/family1 2023-11-28 03:55:16,590 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=41203] regionserver.HStore(775): HFile bounds: first=1\x00\x00\x00\x01 last=1\x00\x00\x00\x07 2023-11-28 03:55:16,590 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=41203] regionserver.HStore(777): Region bounds: first=1 last=2 2023-11-28 03:55:16,590 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41203] regionserver.HStore(775): HFile bounds: first=0\x00\x00\x00\x01 last=0\x00\x00\x00\x07 2023-11-28 03:55:16,590 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41203] regionserver.HStore(777): Region bounds: first= last=1 2023-11-28 03:55:16,590 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=41203] regionserver.HRegion(2446): Flush status journal for b3c2fdb68f55a3cbe8d29ee7b6ddd5e5: 2023-11-28 03:55:16,590 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HStore(775): HFile bounds: first=2\x00\x00\x00\x01 last=2\x00\x00\x00\x07 2023-11-28 03:55:16,590 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HStore(777): Region bounds: first=2 last= 2023-11-28 03:55:16,590 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41203] regionserver.HRegion(2446): Flush status journal for 3d7f49de72f2e9e741f04714c6a7648c: 2023-11-28 03:55:16,591 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HRegion(2446): Flush status journal for 2aad05dc9c73dcbea66259e5c0a6958f: 2023-11-28 03:55:16,592 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=41203] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20231128/family1/da2ed9d2a4bb40cc94bb3d5f1565feb3 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testCompactionWithHFileLink__st2tgmjgujf77ed72pmg8riohofndo13urjatu66e2r57aa689h0qvr8r2bccljn/family1/da2ed9d2a4bb40cc94bb3d5f1565feb3 2023-11-28 03:55:16,592 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20231128/family1/f7aa4e01c9d249ce8ea68c5354267bc0 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testCompactionWithHFileLink__bg49g82rtn0vpcga9cvip04bnjfdpdoii0nsilsv3a3ief203k9up3o96g6e80ed/family1/f7aa4e01c9d249ce8ea68c5354267bc0 2023-11-28 03:55:16,592 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41203] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20231128/family1/ffcdb65d86ed4f4187cb31ec0df9e2fa to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testCompactionWithHFileLink__pmh8hbrg8bg61hvqq67kvn6bonjtslp6fk99v914g2n6v7568pmrcui8mh6rf13i/family1/ffcdb65d86ed4f4187cb31ec0df9e2fa 2023-11-28 03:55:16,596 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=41203] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testCompactionWithHFileLink__st2tgmjgujf77ed72pmg8riohofndo13urjatu66e2r57aa689h0qvr8r2bccljn/family1/da2ed9d2a4bb40cc94bb3d5f1565feb3 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family1/67c4337f7a424469ae26c6084ff7da42_SeqId_29_ 2023-11-28 03:55:16,597 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41203] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testCompactionWithHFileLink__pmh8hbrg8bg61hvqq67kvn6bonjtslp6fk99v914g2n6v7568pmrcui8mh6rf13i/family1/ffcdb65d86ed4f4187cb31ec0df9e2fa as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family1/9c85e4a9998a42e797d2d126f730c0a6_SeqId_29_ 2023-11-28 03:55:16,597 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testCompactionWithHFileLink__bg49g82rtn0vpcga9cvip04bnjfdpdoii0nsilsv3a3ief203k9up3o96g6e80ed/family1/f7aa4e01c9d249ce8ea68c5354267bc0 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family1/caae163a11894964b761ab2b30894b6d_SeqId_29_ 2023-11-28 03:55:16,598 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=41203] regionserver.HStore(849): Loaded HFile hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testCompactionWithHFileLink__st2tgmjgujf77ed72pmg8riohofndo13urjatu66e2r57aa689h0qvr8r2bccljn/family1/da2ed9d2a4bb40cc94bb3d5f1565feb3 into b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family1 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family1/67c4337f7a424469ae26c6084ff7da42_SeqId_29_ - updating store file list. 2023-11-28 03:55:16,598 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HStore(849): Loaded HFile hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testCompactionWithHFileLink__bg49g82rtn0vpcga9cvip04bnjfdpdoii0nsilsv3a3ief203k9up3o96g6e80ed/family1/f7aa4e01c9d249ce8ea68c5354267bc0 into 2aad05dc9c73dcbea66259e5c0a6958f/family1 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family1/caae163a11894964b761ab2b30894b6d_SeqId_29_ - updating store file list. 2023-11-28 03:55:16,598 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41203] regionserver.HStore(849): Loaded HFile hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testCompactionWithHFileLink__pmh8hbrg8bg61hvqq67kvn6bonjtslp6fk99v914g2n6v7568pmrcui8mh6rf13i/family1/ffcdb65d86ed4f4187cb31ec0df9e2fa into 3d7f49de72f2e9e741f04714c6a7648c/family1 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family1/9c85e4a9998a42e797d2d126f730c0a6_SeqId_29_ - updating store file list. 2023-11-28 03:55:16,603 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=41203] regionserver.HStore(882): Loaded HFile hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family1/67c4337f7a424469ae26c6084ff7da42_SeqId_29_ into b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family1 2023-11-28 03:55:16,603 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=41203] regionserver.HStore(855): Successfully loaded hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testCompactionWithHFileLink__st2tgmjgujf77ed72pmg8riohofndo13urjatu66e2r57aa689h0qvr8r2bccljn/family1/da2ed9d2a4bb40cc94bb3d5f1565feb3 into b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family1 (new location: hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family1/67c4337f7a424469ae26c6084ff7da42_SeqId_29_) 2023-11-28 03:55:16,603 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HStore(882): Loaded HFile hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family1/caae163a11894964b761ab2b30894b6d_SeqId_29_ into 2aad05dc9c73dcbea66259e5c0a6958f/family1 2023-11-28 03:55:16,604 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HStore(855): Successfully loaded hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testCompactionWithHFileLink__bg49g82rtn0vpcga9cvip04bnjfdpdoii0nsilsv3a3ief203k9up3o96g6e80ed/family1/f7aa4e01c9d249ce8ea68c5354267bc0 into 2aad05dc9c73dcbea66259e5c0a6958f/family1 (new location: hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family1/caae163a11894964b761ab2b30894b6d_SeqId_29_) 2023-11-28 03:55:16,604 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=41203] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testCompactionWithHFileLink__st2tgmjgujf77ed72pmg8riohofndo13urjatu66e2r57aa689h0qvr8r2bccljn/family1/da2ed9d2a4bb40cc94bb3d5f1565feb3 2023-11-28 03:55:16,604 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testCompactionWithHFileLink__bg49g82rtn0vpcga9cvip04bnjfdpdoii0nsilsv3a3ief203k9up3o96g6e80ed/family1/f7aa4e01c9d249ce8ea68c5354267bc0 2023-11-28 03:55:16,605 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41203] regionserver.HStore(882): Loaded HFile hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family1/9c85e4a9998a42e797d2d126f730c0a6_SeqId_29_ into 3d7f49de72f2e9e741f04714c6a7648c/family1 2023-11-28 03:55:16,605 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41203] regionserver.HStore(855): Successfully loaded hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testCompactionWithHFileLink__pmh8hbrg8bg61hvqq67kvn6bonjtslp6fk99v914g2n6v7568pmrcui8mh6rf13i/family1/ffcdb65d86ed4f4187cb31ec0df9e2fa into 3d7f49de72f2e9e741f04714c6a7648c/family1 (new location: hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family1/9c85e4a9998a42e797d2d126f730c0a6_SeqId_29_) 2023-11-28 03:55:16,605 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41203] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testCompactionWithHFileLink__pmh8hbrg8bg61hvqq67kvn6bonjtslp6fk99v914g2n6v7568pmrcui8mh6rf13i/family1/ffcdb65d86ed4f4187cb31ec0df9e2fa 2023-11-28 03:55:16,607 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20231128 table='testCompactionWithHFileLink' column='family1' 2023-11-28 03:55:16,607 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20231128 table='testCompactionWithHFileLink' column='family1' 2023-11-28 03:55:16,608 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(360): Archiving compacted files. 2023-11-28 03:55:16,608 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(360): Archiving compacted files. 2023-11-28 03:55:16,608 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20231128 table='testCompactionWithHFileLink' column='family1' 2023-11-28 03:55:16,609 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-28 03:55:16,609 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20231128548ceb87df594da68e7f4d2121a8bbf7 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20231128548ceb87df594da68e7f4d2121a8bbf7 2023-11-28 03:55:16,610 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023112838c9c1bb8cde4e7f815926db3a63a195 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023112838c9c1bb8cde4e7f815926db3a63a195 2023-11-28 03:55:16,611 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023112878e8d44c7cc44d39bafe2620db40b1d7 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023112878e8d44c7cc44d39bafe2620db40b1d7 2023-11-28 03:55:16,611 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20231128dec752e8fb644a0a80d6b4fd043402c1 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20231128dec752e8fb644a0a80d6b4fd043402c1 2023-11-28 03:55:16,611 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202311280eef9bc4c106435986d8700c2b7868aa to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202311280eef9bc4c106435986d8700c2b7868aa 2023-11-28 03:55:16,613 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202311287fb879a012e040c992a5c4c12e52bff2 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202311287fb879a012e040c992a5c4c12e52bff2 2023-11-28 03:55:16,613 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20231128eb89ca1113e144ddaad3b48ac062b47f to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20231128eb89ca1113e144ddaad3b48ac062b47f 2023-11-28 03:55:16,614 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202311285c06394279144f2aa2ca5a78e6ddb299 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202311285c06394279144f2aa2ca5a78e6ddb299 2023-11-28 03:55:16,615 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20231128bd3184bcc4744bf591f18e1d0b9b0eb7 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20231128bd3184bcc4744bf591f18e1d0b9b0eb7 2023-11-28 03:55:16,615 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-11-28 03:55:16,615 DEBUG [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20231128' 2023-11-28 03:55:16,615 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20231128fb09572884984543b4a9faae21b5679c to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20231128fb09572884984543b4a9faae21b5679c 2023-11-28 03:55:16,615 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c4ca4238a0b923820dcc509a6f75849b20231128 for table='testCompactionWithHFileLink' and column='family1' 2023-11-28 03:55:16,615 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e2023112864ca880632ff4b81b856e56f97a51ba5 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e2023112864ca880632ff4b81b856e56f97a51ba5 2023-11-28 03:55:16,615 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-11-28 03:55:16,616 DEBUG [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20231128' 2023-11-28 03:55:16,616 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c81e728d9d4c2f636f067f89cc14862c20231128 for table='testCompactionWithHFileLink' and column='family1' 2023-11-28 03:55:16,617 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20231128ae21d695d78b47819031a63d23fd1adf to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20231128ae21d695d78b47819031a63d23fd1adf 2023-11-28 03:55:16,617 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-11-28 03:55:16,617 DEBUG [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20231128' 2023-11-28 03:55:16,617 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20231128 for table='testCompactionWithHFileLink' and column='family1' 2023-11-28 03:55:16,618 INFO [Listener at localhost/42853] compactions.PartitionedMobCompactor(354): After compaction, there are 3 mob files. table='testCompactionWithHFileLink' column='family1' 2023-11-28 03:55:16,618 INFO [Listener at localhost/42853] compactions.PartitionedMobCompactor(364): After a mob compaction with all files selected, archiving the del files for table='testCompactionWithHFileLink' and column='family1' 2023-11-28 03:55:16,618 INFO [Listener at localhost/42853] compactions.PartitionedMobCompactor(367): [hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20231128e3e9d5da083145f088ada4b38be067fe_del] 2023-11-28 03:55:16,618 DEBUG [Listener at localhost/42853] backup.HFileArchiver(360): Archiving compacted files. 2023-11-28 03:55:16,620 DEBUG [Listener at localhost/42853] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20231128e3e9d5da083145f088ada4b38be067fe_del to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20231128e3e9d5da083145f088ada4b38be067fe_del 2023-11-28 03:55:16,620 INFO [Listener at localhost/42853] mob.MobUtils(508): Table testCompactionWithHFileLink 1 expired mob files are deleted 2023-11-28 03:55:16,620 INFO [Listener at localhost/42853] compactions.PartitionedMobCompactor(367): [hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023112804c33ea8ae7e4de4aec70160568ac534_del] 2023-11-28 03:55:16,621 DEBUG [Listener at localhost/42853] backup.HFileArchiver(360): Archiving compacted files. 2023-11-28 03:55:16,622 DEBUG [Listener at localhost/42853] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023112804c33ea8ae7e4de4aec70160568ac534_del to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023112804c33ea8ae7e4de4aec70160568ac534_del 2023-11-28 03:55:16,622 INFO [Listener at localhost/42853] mob.MobUtils(508): Table testCompactionWithHFileLink 1 expired mob files are deleted 2023-11-28 03:55:16,622 INFO [Listener at localhost/42853] compactions.PartitionedMobCompactor(367): [hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023112884b65916c9774d4c8416f134fef2f12f_del] 2023-11-28 03:55:16,623 DEBUG [Listener at localhost/42853] backup.HFileArchiver(360): Archiving compacted files. 2023-11-28 03:55:16,624 DEBUG [Listener at localhost/42853] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023112884b65916c9774d4c8416f134fef2f12f_del to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023112884b65916c9774d4c8416f134fef2f12f_del 2023-11-28 03:55:16,624 INFO [Listener at localhost/42853] mob.MobUtils(508): Table testCompactionWithHFileLink 1 expired mob files are deleted 2023-11-28 03:55:16,628 DEBUG [Listener at localhost/42853] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-11-28 03:55:16,631 DEBUG [Listener at localhost/42853] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-11-28 03:55:16,634 DEBUG [Listener at localhost/42853] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-11-28 03:55:16,635 DEBUG [Listener at localhost/42853] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-11-28 03:55:16,641 INFO [Listener at localhost/42853] client.HBaseAdmin$15(890): Started disable of testCompactionWithHFileLink 2023-11-28 03:55:16,641 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] master.HMaster$11(2418): Client=jenkins//172.31.14.131 disable testCompactionWithHFileLink 2023-11-28 03:55:16,642 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] procedure2.ProcedureExecutor(1028): Stored pid=88, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testCompactionWithHFileLink 2023-11-28 03:55:16,644 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] master.MasterRpcServices(1230): Checking to see if procedure is done pid=88 2023-11-28 03:55:16,645 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1701143716645"}]},"ts":"1701143716645"} 2023-11-28 03:55:16,646 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=DISABLING in hbase:meta 2023-11-28 03:55:16,648 INFO [PEWorker-5] procedure.DisableTableProcedure(293): Set testCompactionWithHFileLink to state=DISABLING 2023-11-28 03:55:16,648 INFO [PEWorker-5] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=89, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=3d7f49de72f2e9e741f04714c6a7648c, UNASSIGN}, {pid=90, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=b3c2fdb68f55a3cbe8d29ee7b6ddd5e5, UNASSIGN}, {pid=91, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=2aad05dc9c73dcbea66259e5c0a6958f, UNASSIGN}] 2023-11-28 03:55:16,650 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=90, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=b3c2fdb68f55a3cbe8d29ee7b6ddd5e5, UNASSIGN 2023-11-28 03:55:16,650 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=91, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=2aad05dc9c73dcbea66259e5c0a6958f, UNASSIGN 2023-11-28 03:55:16,650 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=89, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=3d7f49de72f2e9e741f04714c6a7648c, UNASSIGN 2023-11-28 03:55:16,651 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=b3c2fdb68f55a3cbe8d29ee7b6ddd5e5, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:16,651 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=91 updating hbase:meta row=2aad05dc9c73dcbea66259e5c0a6958f, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:16,651 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=89 updating hbase:meta row=3d7f49de72f2e9e741f04714c6a7648c, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:16,651 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,2,1701143695226.2aad05dc9c73dcbea66259e5c0a6958f.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1701143716651"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1701143716651"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1701143716651"}]},"ts":"1701143716651"} 2023-11-28 03:55:16,651 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,1,1701143695226.b3c2fdb68f55a3cbe8d29ee7b6ddd5e5.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1701143716651"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1701143716651"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1701143716651"}]},"ts":"1701143716651"} 2023-11-28 03:55:16,651 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,,1701143695226.3d7f49de72f2e9e741f04714c6a7648c.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1701143716651"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1701143716651"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1701143716651"}]},"ts":"1701143716651"} 2023-11-28 03:55:16,652 INFO [PEWorker-2] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=92, ppid=91, state=RUNNABLE; CloseRegionProcedure 2aad05dc9c73dcbea66259e5c0a6958f, server=jenkins-hbase4.apache.org,41203,1701143600653}] 2023-11-28 03:55:16,653 INFO [PEWorker-5] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=93, ppid=90, state=RUNNABLE; CloseRegionProcedure b3c2fdb68f55a3cbe8d29ee7b6ddd5e5, server=jenkins-hbase4.apache.org,41203,1701143600653}] 2023-11-28 03:55:16,654 INFO [PEWorker-4] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=94, ppid=89, state=RUNNABLE; CloseRegionProcedure 3d7f49de72f2e9e741f04714c6a7648c, server=jenkins-hbase4.apache.org,41203,1701143600653}] 2023-11-28 03:55:16,745 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] master.MasterRpcServices(1230): Checking to see if procedure is done pid=88 2023-11-28 03:55:16,804 DEBUG [RSProcedureDispatcher-pool-1] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:16,804 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 3d7f49de72f2e9e741f04714c6a7648c 2023-11-28 03:55:16,804 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 3d7f49de72f2e9e741f04714c6a7648c, disabling compactions & flushes 2023-11-28 03:55:16,804 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,,1701143695226.3d7f49de72f2e9e741f04714c6a7648c. 2023-11-28 03:55:16,804 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,,1701143695226.3d7f49de72f2e9e741f04714c6a7648c. 2023-11-28 03:55:16,804 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,,1701143695226.3d7f49de72f2e9e741f04714c6a7648c. after waiting 0 ms 2023-11-28 03:55:16,804 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,,1701143695226.3d7f49de72f2e9e741f04714c6a7648c. 2023-11-28 03:55:16,809 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/recovered.edits/31.seqid, newMaxSeqId=31, maxSeqId=1 2023-11-28 03:55:16,810 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,,1701143695226.3d7f49de72f2e9e741f04714c6a7648c. 2023-11-28 03:55:16,810 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 3d7f49de72f2e9e741f04714c6a7648c: 2023-11-28 03:55:16,812 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 3d7f49de72f2e9e741f04714c6a7648c 2023-11-28 03:55:16,812 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 2aad05dc9c73dcbea66259e5c0a6958f 2023-11-28 03:55:16,812 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 2aad05dc9c73dcbea66259e5c0a6958f, disabling compactions & flushes 2023-11-28 03:55:16,812 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,2,1701143695226.2aad05dc9c73dcbea66259e5c0a6958f. 2023-11-28 03:55:16,812 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,2,1701143695226.2aad05dc9c73dcbea66259e5c0a6958f. 2023-11-28 03:55:16,812 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,2,1701143695226.2aad05dc9c73dcbea66259e5c0a6958f. after waiting 0 ms 2023-11-28 03:55:16,812 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,2,1701143695226.2aad05dc9c73dcbea66259e5c0a6958f. 2023-11-28 03:55:16,812 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=89 updating hbase:meta row=3d7f49de72f2e9e741f04714c6a7648c, regionState=CLOSED 2023-11-28 03:55:16,812 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,,1701143695226.3d7f49de72f2e9e741f04714c6a7648c.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1701143716812"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1701143716812"}]},"ts":"1701143716812"} 2023-11-28 03:55:16,815 INFO [PEWorker-5] procedure2.ProcedureExecutor(1825): Finished subprocedure pid=94, resume processing ppid=89 2023-11-28 03:55:16,815 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=94, ppid=89, state=SUCCESS; CloseRegionProcedure 3d7f49de72f2e9e741f04714c6a7648c, server=jenkins-hbase4.apache.org,41203,1701143600653 in 160 msec 2023-11-28 03:55:16,816 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/recovered.edits/31.seqid, newMaxSeqId=31, maxSeqId=1 2023-11-28 03:55:16,816 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=89, ppid=88, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=3d7f49de72f2e9e741f04714c6a7648c, UNASSIGN in 167 msec 2023-11-28 03:55:16,817 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,2,1701143695226.2aad05dc9c73dcbea66259e5c0a6958f. 2023-11-28 03:55:16,817 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 2aad05dc9c73dcbea66259e5c0a6958f: 2023-11-28 03:55:16,818 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 2aad05dc9c73dcbea66259e5c0a6958f 2023-11-28 03:55:16,818 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close b3c2fdb68f55a3cbe8d29ee7b6ddd5e5 2023-11-28 03:55:16,818 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing b3c2fdb68f55a3cbe8d29ee7b6ddd5e5, disabling compactions & flushes 2023-11-28 03:55:16,818 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,1,1701143695226.b3c2fdb68f55a3cbe8d29ee7b6ddd5e5. 2023-11-28 03:55:16,818 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,1,1701143695226.b3c2fdb68f55a3cbe8d29ee7b6ddd5e5. 2023-11-28 03:55:16,818 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,1,1701143695226.b3c2fdb68f55a3cbe8d29ee7b6ddd5e5. after waiting 0 ms 2023-11-28 03:55:16,818 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,1,1701143695226.b3c2fdb68f55a3cbe8d29ee7b6ddd5e5. 2023-11-28 03:55:16,818 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=91 updating hbase:meta row=2aad05dc9c73dcbea66259e5c0a6958f, regionState=CLOSED 2023-11-28 03:55:16,818 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,2,1701143695226.2aad05dc9c73dcbea66259e5c0a6958f.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1701143716818"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1701143716818"}]},"ts":"1701143716818"} 2023-11-28 03:55:16,821 INFO [PEWorker-4] procedure2.ProcedureExecutor(1825): Finished subprocedure pid=92, resume processing ppid=91 2023-11-28 03:55:16,821 INFO [PEWorker-4] procedure2.ProcedureExecutor(1409): Finished pid=92, ppid=91, state=SUCCESS; CloseRegionProcedure 2aad05dc9c73dcbea66259e5c0a6958f, server=jenkins-hbase4.apache.org,41203,1701143600653 in 168 msec 2023-11-28 03:55:16,822 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/recovered.edits/31.seqid, newMaxSeqId=31, maxSeqId=1 2023-11-28 03:55:16,822 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=91, ppid=88, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=2aad05dc9c73dcbea66259e5c0a6958f, UNASSIGN in 173 msec 2023-11-28 03:55:16,823 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,1,1701143695226.b3c2fdb68f55a3cbe8d29ee7b6ddd5e5. 2023-11-28 03:55:16,823 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for b3c2fdb68f55a3cbe8d29ee7b6ddd5e5: 2023-11-28 03:55:16,824 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed b3c2fdb68f55a3cbe8d29ee7b6ddd5e5 2023-11-28 03:55:16,824 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=b3c2fdb68f55a3cbe8d29ee7b6ddd5e5, regionState=CLOSED 2023-11-28 03:55:16,824 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,1,1701143695226.b3c2fdb68f55a3cbe8d29ee7b6ddd5e5.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1701143716824"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1701143716824"}]},"ts":"1701143716824"} 2023-11-28 03:55:16,826 INFO [PEWorker-3] procedure2.ProcedureExecutor(1825): Finished subprocedure pid=93, resume processing ppid=90 2023-11-28 03:55:16,826 INFO [PEWorker-3] procedure2.ProcedureExecutor(1409): Finished pid=93, ppid=90, state=SUCCESS; CloseRegionProcedure b3c2fdb68f55a3cbe8d29ee7b6ddd5e5, server=jenkins-hbase4.apache.org,41203,1701143600653 in 173 msec 2023-11-28 03:55:16,827 INFO [PEWorker-5] procedure2.ProcedureExecutor(1825): Finished subprocedure pid=90, resume processing ppid=88 2023-11-28 03:55:16,827 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=90, ppid=88, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=b3c2fdb68f55a3cbe8d29ee7b6ddd5e5, UNASSIGN in 178 msec 2023-11-28 03:55:16,827 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1701143716827"}]},"ts":"1701143716827"} 2023-11-28 03:55:16,828 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=DISABLED in hbase:meta 2023-11-28 03:55:16,831 INFO [PEWorker-2] procedure.DisableTableProcedure(305): Set testCompactionWithHFileLink to state=DISABLED 2023-11-28 03:55:16,832 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=88, state=SUCCESS; DisableTableProcedure table=testCompactionWithHFileLink in 190 msec 2023-11-28 03:55:16,947 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] master.MasterRpcServices(1230): Checking to see if procedure is done pid=88 2023-11-28 03:55:16,947 INFO [Listener at localhost/42853] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testCompactionWithHFileLink, procId: 88 completed 2023-11-28 03:55:16,957 INFO [Listener at localhost/42853] client.HBaseAdmin(2586): Taking restore-failsafe snapshot: hbase-failsafe-snaptb-1701143704146-1701143716957 2023-11-28 03:55:16,958 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] master.MasterRpcServices(1583): Client=jenkins//172.31.14.131 snapshot request for:{ ss=hbase-failsafe-snaptb-1701143704146-1701143716957 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-11-28 03:55:16,958 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] snapshot.SnapshotDescriptionUtils(316): Creation time not specified, setting to:1701143716958 (current time:1701143716958). 2023-11-28 03:55:16,958 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] snapshot.SnapshotDescriptionUtils(332): Snapshot current TTL value: 0 resetting it to default value: 0 2023-11-28 03:55:16,959 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] zookeeper.ReadOnlyZKClient(139): Connect 0x2e944f10 to 127.0.0.1:55641 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-11-28 03:55:16,963 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] ipc.AbstractRpcClient(189): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@79b632c7, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-11-28 03:55:16,964 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] ipc.RpcConnection(122): Using SIMPLE authentication for service=ClientService, sasl=false 2023-11-28 03:55:16,966 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:55318, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-11-28 03:55:16,967 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x2e944f10 to 127.0.0.1:55641 2023-11-28 03:55:16,967 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] ipc.AbstractRpcClient(489): Stopping rpc client 2023-11-28 03:55:16,967 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] snapshot.SnapshotManager(637): No existing snapshot, attempting snapshot... 2023-11-28 03:55:16,967 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] snapshot.SnapshotManager(695): Table is disabled, running snapshot entirely on master for { ss=hbase-failsafe-snaptb-1701143704146-1701143716957 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-11-28 03:55:16,970 WARN [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] snapshot.SnapshotManager(519): Unable to retrieve ACL status for path: hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/.hbase-snapshot, current working dir path: hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/.hbase-snapshot/.tmp/hbase-failsafe-snaptb-1701143704146-1701143716957 org.apache.hadoop.hdfs.protocol.AclException: The ACL operation has been rejected. Support for ACLs has been disabled by setting dfs.namenode.acls.enabled to false. at org.apache.hadoop.hdfs.server.namenode.FSDirAclOp.checkAclsConfigFlag(FSDirAclOp.java:197) at org.apache.hadoop.hdfs.server.namenode.FSDirAclOp.getAclStatus(FSDirAclOp.java:150) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAclStatus(FSNamesystem.java:6744) at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.getAclStatus(NameNodeRpcServer.java:2065) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.getAclStatus(ClientNamenodeProtocolServerSideTranslatorPB.java:1378) at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java) at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:507) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1034) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:994) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:922) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1893) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2833) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:121) at org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:88) at org.apache.hadoop.hdfs.DFSClient.getAclStatus(DFSClient.java:2736) at org.apache.hadoop.hdfs.DistributedFileSystem$51.doCall(DistributedFileSystem.java:2294) at org.apache.hadoop.hdfs.DistributedFileSystem$51.doCall(DistributedFileSystem.java:2291) at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81) at org.apache.hadoop.hdfs.DistributedFileSystem.getAclStatus(DistributedFileSystem.java:2291) at org.apache.hadoop.hbase.master.snapshot.SnapshotManager.updateWorkingDirAclsIfRequired(SnapshotManager.java:517) at org.apache.hadoop.hbase.master.snapshot.SnapshotManager.prepareToTakeSnapshot(SnapshotManager.java:490) at org.apache.hadoop.hbase.master.snapshot.SnapshotManager.snapshotDisabledTable(SnapshotManager.java:541) at org.apache.hadoop.hbase.master.snapshot.SnapshotManager.takeSnapshotInternal(SnapshotManager.java:698) at org.apache.hadoop.hbase.master.snapshot.SnapshotManager.takeSnapshot(SnapshotManager.java:623) at org.apache.hadoop.hbase.master.MasterRpcServices.snapshot(MasterRpcServices.java:1588) at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) Caused by: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.hdfs.protocol.AclException): The ACL operation has been rejected. Support for ACLs has been disabled by setting dfs.namenode.acls.enabled to false. at org.apache.hadoop.hdfs.server.namenode.FSDirAclOp.checkAclsConfigFlag(FSDirAclOp.java:197) at org.apache.hadoop.hdfs.server.namenode.FSDirAclOp.getAclStatus(FSDirAclOp.java:150) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAclStatus(FSNamesystem.java:6744) at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.getAclStatus(NameNodeRpcServer.java:2065) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.getAclStatus(ClientNamenodeProtocolServerSideTranslatorPB.java:1378) at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java) at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:507) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1034) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:994) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:922) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1893) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2833) at org.apache.hadoop.ipc.Client.getRpcResponse(Client.java:1540) at org.apache.hadoop.ipc.Client.call(Client.java:1486) at org.apache.hadoop.ipc.Client.call(Client.java:1385) at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:232) at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:118) at com.sun.proxy.$Proxy29.getAclStatus(Unknown Source) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getAclStatus(ClientNamenodeProtocolTranslatorPB.java:1385) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:422) at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeMethod(RetryInvocationHandler.java:165) at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invoke(RetryInvocationHandler.java:157) at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeOnce(RetryInvocationHandler.java:95) at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:359) at com.sun.proxy.$Proxy30.getAclStatus(Unknown Source) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.getAclStatus(Unknown Source) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.getAclStatus(Unknown Source) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.getAclStatus(Unknown Source) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.getAclStatus(Unknown Source) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.getAclStatus(Unknown Source) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.getAclStatus(Unknown Source) at org.apache.hadoop.hdfs.DFSClient.getAclStatus(DFSClient.java:2734) ... 15 more 2023-11-28 03:55:16,971 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] procedure2.ProcedureExecutor(1028): Stored pid=95, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-11-28 03:55:16,972 DEBUG [PEWorker-4] locking.LockProcedure(309): LOCKED pid=95, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-11-28 03:55:16,973 INFO [PEWorker-4] procedure2.TimeoutExecutorThread(81): ADDED pid=95, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE; timeout=600000, timestamp=1701144316973 2023-11-28 03:55:16,973 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(174): Running DISABLED table snapshot hbase-failsafe-snaptb-1701143704146-1701143716957 C_M_SNAPSHOT_TABLE on table testCompactionWithHFileLink 2023-11-28 03:55:16,973 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] snapshot.SnapshotManager(700): Started snapshot: { ss=hbase-failsafe-snaptb-1701143704146-1701143716957 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-11-28 03:55:16,973 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2418): Waiting a max of 300000 ms for snapshot '{ ss=hbase-failsafe-snaptb-1701143704146-1701143716957 table=testCompactionWithHFileLink type=FLUSH ttl=0 }'' to complete. (max 300000 ms per retry) 2023-11-28 03:55:16,973 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2428): (#1) Sleeping: 100ms while waiting for snapshot completion. 2023-11-28 03:55:17,073 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-11-28 03:55:17,075 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] master.MasterRpcServices(1212): Checking to see if snapshot from request:{ ss=hbase-failsafe-snaptb-1701143704146-1701143716957 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-11-28 03:55:17,075 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] snapshot.SnapshotManager(406): Snapshoting '{ ss=hbase-failsafe-snaptb-1701143704146-1701143716957 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-11-28 03:55:17,075 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2428): (#2) Sleeping: 200ms while waiting for snapshot completion. 2023-11-28 03:55:17,275 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-11-28 03:55:17,276 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] master.MasterRpcServices(1212): Checking to see if snapshot from request:{ ss=hbase-failsafe-snaptb-1701143704146-1701143716957 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-11-28 03:55:17,276 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] snapshot.SnapshotManager(406): Snapshoting '{ ss=hbase-failsafe-snaptb-1701143704146-1701143716957 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-11-28 03:55:17,276 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2428): (#3) Sleeping: 300ms while waiting for snapshot completion. 2023-11-28 03:55:17,379 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.DisabledTableSnapshotHandler(97): Starting to write region info and WALs for regions for offline snapshot:{ ss=hbase-failsafe-snaptb-1701143704146-1701143716957 table=testCompactionWithHFileLink type=DISABLED ttl=0 } 2023-11-28 03:55:17,381 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(297): Storing region-info for snapshot. 2023-11-28 03:55:17,381 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(297): Storing region-info for snapshot. 2023-11-28 03:55:17,381 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(297): Storing region-info for snapshot. 2023-11-28 03:55:17,381 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(302): Creating references for hfiles 2023-11-28 03:55:17,381 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(297): Storing region-info for snapshot. 2023-11-28 03:55:17,381 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(302): Creating references for hfiles 2023-11-28 03:55:17,381 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(302): Creating references for hfiles 2023-11-28 03:55:17,381 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(302): Creating references for hfiles 2023-11-28 03:55:17,384 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family1/2c88d564dad043448d1c89f73401463e, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family1/67c4337f7a424469ae26c6084ff7da42_SeqId_29_] hfiles 2023-11-28 03:55:17,384 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(363): Adding reference for hfile (1/2): hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family1/2c88d564dad043448d1c89f73401463e 2023-11-28 03:55:17,384 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(363): Adding reference for hfile (2/2): hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family1/67c4337f7a424469ae26c6084ff7da42_SeqId_29_ 2023-11-28 03:55:17,384 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family1/2c5387a00c534c1ba4b9b3b4a82a16db, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family1/9c85e4a9998a42e797d2d126f730c0a6_SeqId_29_] hfiles 2023-11-28 03:55:17,384 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (1/2): hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family1/2c5387a00c534c1ba4b9b3b4a82a16db 2023-11-28 03:55:17,384 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family1/b54918ba975e4d51b6b332b5e3e581de, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family1/caae163a11894964b761ab2b30894b6d_SeqId_29_] hfiles 2023-11-28 03:55:17,384 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (2/2): hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family1/9c85e4a9998a42e797d2d126f730c0a6_SeqId_29_ 2023-11-28 03:55:17,384 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(363): Adding reference for hfile (1/2): hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family1/b54918ba975e4d51b6b332b5e3e581de 2023-11-28 03:55:17,384 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(363): Adding reference for hfile (2/2): hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family1/caae163a11894964b761ab2b30894b6d_SeqId_29_ 2023-11-28 03:55:17,385 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20231128e981aa212a084c27a0d588aba8fd58d9, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20231128d2c59f3654234b0db8ace0004962cc6d, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20231128ff787fef44d04a7e9bac61acbfca183b] hfiles 2023-11-28 03:55:17,385 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (1/3): hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20231128e981aa212a084c27a0d588aba8fd58d9 2023-11-28 03:55:17,385 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family2/c4c67664540a48089e64216c5442c7b5] hfiles 2023-11-28 03:55:17,385 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (2/3): hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20231128d2c59f3654234b0db8ace0004962cc6d 2023-11-28 03:55:17,385 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(363): Adding reference for hfile (1/1): hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family2/c4c67664540a48089e64216c5442c7b5 2023-11-28 03:55:17,385 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (3/3): hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20231128ff787fef44d04a7e9bac61acbfca183b 2023-11-28 03:55:17,385 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family2/79ff143814eb45a58e52ea79560b8f93] hfiles 2023-11-28 03:55:17,385 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (1/1): hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family2/79ff143814eb45a58e52ea79560b8f93 2023-11-28 03:55:17,385 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family2/b93f7d6b9dbf4058937d8aa40eb41bb5] hfiles 2023-11-28 03:55:17,385 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(363): Adding reference for hfile (1/1): hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family2/b93f7d6b9dbf4058937d8aa40eb41bb5 2023-11-28 03:55:17,391 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202311282ac64a54290b4a59b37205d583268af5, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202311283312fc03dd134b7bab571a71f58de4a3, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b2023112847e6b2a73e5648f988213810b241c45a_del, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202311284dec3a2023324667bf8796b80a2f3bce, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b2023112855e11b7d60d0483689ffc29e4b1a402e, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202311284eac4cd4063e44bf9de8ec54912c3ff0, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202311284edd51e4fb9f46e894cf420c094d1251, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023112889097115e6d245cc97ff3c92d83f537e, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20231128f02c18cd45f44d048d76917a7a311c0a, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20231128f2a1b19354fb4a1ea87f959529de2a75_del, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202311281197d88dc8b0498eb25fac89897a29de, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20231128203e460f7da34a628c95173a08d56f3b_del, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202311282cafb2c5fdbd4fcf9ce84de6f28bfe1b, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20231128310f916e533545709f94215002f0c70b, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202311284073e83e8d4242548321ae15f6b216d4] hfiles 2023-11-28 03:55:17,391 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (1/15): hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202311282ac64a54290b4a59b37205d583268af5 2023-11-28 03:55:17,391 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (2/15): hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202311283312fc03dd134b7bab571a71f58de4a3 2023-11-28 03:55:17,391 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (3/15): hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b2023112847e6b2a73e5648f988213810b241c45a_del 2023-11-28 03:55:17,391 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (4/15): hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202311284dec3a2023324667bf8796b80a2f3bce 2023-11-28 03:55:17,391 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (5/15): hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b2023112855e11b7d60d0483689ffc29e4b1a402e 2023-11-28 03:55:17,391 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (6/15): hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202311284eac4cd4063e44bf9de8ec54912c3ff0 2023-11-28 03:55:17,391 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (7/15): hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202311284edd51e4fb9f46e894cf420c094d1251 2023-11-28 03:55:17,391 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (8/15): hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023112889097115e6d245cc97ff3c92d83f537e 2023-11-28 03:55:17,391 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (9/15): hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20231128f02c18cd45f44d048d76917a7a311c0a 2023-11-28 03:55:17,391 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (10/15): hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20231128f2a1b19354fb4a1ea87f959529de2a75_del 2023-11-28 03:55:17,391 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (11/15): hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202311281197d88dc8b0498eb25fac89897a29de 2023-11-28 03:55:17,392 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (12/15): hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20231128203e460f7da34a628c95173a08d56f3b_del 2023-11-28 03:55:17,392 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (13/15): hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202311282cafb2c5fdbd4fcf9ce84de6f28bfe1b 2023-11-28 03:55:17,392 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (14/15): hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20231128310f916e533545709f94215002f0c70b 2023-11-28 03:55:17,392 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (15/15): hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202311284073e83e8d4242548321ae15f6b216d4 2023-11-28 03:55:17,576 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-11-28 03:55:17,577 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] master.MasterRpcServices(1212): Checking to see if snapshot from request:{ ss=hbase-failsafe-snaptb-1701143704146-1701143716957 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-11-28 03:55:17,577 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] snapshot.SnapshotManager(406): Snapshoting '{ ss=hbase-failsafe-snaptb-1701143704146-1701143716957 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-11-28 03:55:17,577 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2428): (#4) Sleeping: 500ms while waiting for snapshot completion. 2023-11-28 03:55:17,796 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.DisabledTableSnapshotHandler(120): Marking snapshot{ ss=hbase-failsafe-snaptb-1701143704146-1701143716957 table=testCompactionWithHFileLink type=DISABLED ttl=0 } as finished. 2023-11-28 03:55:17,796 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(484): Convert to Single Snapshot Manifest for hbase-failsafe-snaptb-1701143704146-1701143716957 2023-11-28 03:55:17,797 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifestV1(126): No regions under directory:hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/.hbase-snapshot/.tmp/hbase-failsafe-snaptb-1701143704146-1701143716957 2023-11-28 03:55:18,078 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-11-28 03:55:18,078 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] master.MasterRpcServices(1212): Checking to see if snapshot from request:{ ss=hbase-failsafe-snaptb-1701143704146-1701143716957 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-11-28 03:55:18,078 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] snapshot.SnapshotManager(406): Snapshoting '{ ss=hbase-failsafe-snaptb-1701143704146-1701143716957 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-11-28 03:55:18,079 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2428): (#5) Sleeping: 1000ms while waiting for snapshot completion. 2023-11-28 03:55:18,221 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotDescriptionUtils(404): Sentinel is done, just moving the snapshot from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/.hbase-snapshot/.tmp/hbase-failsafe-snaptb-1701143704146-1701143716957 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/.hbase-snapshot/hbase-failsafe-snaptb-1701143704146-1701143716957 2023-11-28 03:55:18,222 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(229): Snapshot hbase-failsafe-snaptb-1701143704146-1701143716957 of table testCompactionWithHFileLink completed 2023-11-28 03:55:18,222 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(246): Launching cleanup of working dir:hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/.hbase-snapshot/.tmp/hbase-failsafe-snaptb-1701143704146-1701143716957 2023-11-28 03:55:18,222 ERROR [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(251): Couldn't delete snapshot working directory:hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/.hbase-snapshot/.tmp/hbase-failsafe-snaptb-1701143704146-1701143716957 2023-11-28 03:55:18,222 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(257): Table snapshot journal : Running DISABLED table snapshot hbase-failsafe-snaptb-1701143704146-1701143716957 C_M_SNAPSHOT_TABLE on table testCompactionWithHFileLink at 1701143716973Starting to write region info and WALs for regions for offline snapshot:{ ss=hbase-failsafe-snaptb-1701143704146-1701143716957 table=testCompactionWithHFileLink type=DISABLED ttl=0 } at 1701143717379 (+406 ms)Consolidate snapshot: hbase-failsafe-snaptb-1701143704146-1701143716957 at 1701143717796 (+417 ms)Loading Region manifests for hbase-failsafe-snaptb-1701143704146-1701143716957 at 1701143717796Writing data manifest for hbase-failsafe-snaptb-1701143704146-1701143716957 at 1701143717799 (+3 ms)Verifying snapshot: hbase-failsafe-snaptb-1701143704146-1701143716957 at 1701143718205 (+406 ms)Snapshot hbase-failsafe-snaptb-1701143704146-1701143716957 of table testCompactionWithHFileLink completed at 1701143718222 (+17 ms) 2023-11-28 03:55:18,224 DEBUG [PEWorker-1] locking.LockProcedure(242): UNLOCKED pid=95, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-11-28 03:55:18,225 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=95, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE in 1.2530 sec 2023-11-28 03:55:19,079 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-11-28 03:55:19,080 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] master.MasterRpcServices(1212): Checking to see if snapshot from request:{ ss=hbase-failsafe-snaptb-1701143704146-1701143716957 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-11-28 03:55:19,080 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] snapshot.SnapshotManager(403): Snapshot '{ ss=hbase-failsafe-snaptb-1701143704146-1701143716957 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' has completed, notifying client. 2023-11-28 03:55:19,129 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] procedure2.ProcedureExecutor(1028): Stored pid=96, state=RUNNABLE:RESTORE_SNAPSHOT_PRE_OPERATION; RestoreSnapshotProcedure (table=testCompactionWithHFileLink snapshot=name: "snaptb-1701143704146" table: "testCompactionWithHFileLink" creation_time: 1701143704156 type: FLUSH version: 2 ttl: 0 ) 2023-11-28 03:55:19,130 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] snapshot.SnapshotManager(916): Restore snapshot=snaptb-1701143704146 as table=testCompactionWithHFileLink 2023-11-28 03:55:19,133 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] master.MasterRpcServices(1230): Checking to see if procedure is done pid=96 2023-11-28 03:55:19,234 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] master.MasterRpcServices(1230): Checking to see if procedure is done pid=96 2023-11-28 03:55:19,435 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] master.MasterRpcServices(1230): Checking to see if procedure is done pid=96 2023-11-28 03:55:19,543 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/.tabledesc/.tableinfo.0000000002 2023-11-28 03:55:19,544 DEBUG [PEWorker-3] util.FSTableDescriptors(519): Deleted hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/.tabledesc/.tableinfo.0000000001 2023-11-28 03:55:19,544 INFO [PEWorker-3] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/.tabledesc/.tableinfo.0000000002 2023-11-28 03:55:19,545 INFO [PEWorker-3] procedure.RestoreSnapshotProcedure(384): Starting restore snapshot={ ss=snaptb-1701143704146 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-11-28 03:55:19,550 INFO [PEWorker-3] snapshot.RestoreSnapshotHelper(177): starting restore table regions using snapshot=name: "snaptb-1701143704146" table: "testCompactionWithHFileLink" creation_time: 1701143704156 type: FLUSH version: 2 ttl: 0 2023-11-28 03:55:19,551 DEBUG [PEWorker-3] snapshot.RestoreSnapshotHelper(785): get table regions: hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink 2023-11-28 03:55:19,556 DEBUG [PEWorker-3] snapshot.RestoreSnapshotHelper(797): found 3 regions for table=testCompactionWithHFileLink 2023-11-28 03:55:19,559 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20231128d2c59f3654234b0db8ace0004962cc6d to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20231128d2c59f3654234b0db8ace0004962cc6d 2023-11-28 03:55:19,560 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20231128e981aa212a084c27a0d588aba8fd58d9 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20231128e981aa212a084c27a0d588aba8fd58d9 2023-11-28 03:55:19,561 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20231128ff787fef44d04a7e9bac61acbfca183b to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20231128ff787fef44d04a7e9bac61acbfca183b 2023-11-28 03:55:19,561 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c4ca4238a0b923820dcc509a6f75849b2023112838c9c1bb8cde4e7f815926db3a63a195 of snapshot=snaptb-1701143704146 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-11-28 03:55:19,565 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c4ca4238a0b923820dcc509a6f75849b2023112878e8d44c7cc44d39bafe2620db40b1d7 of snapshot=snaptb-1701143704146 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-11-28 03:55:19,568 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c4ca4238a0b923820dcc509a6f75849b202311287fb879a012e040c992a5c4c12e52bff2 of snapshot=snaptb-1701143704146 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-11-28 03:55:19,572 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c4ca4238a0b923820dcc509a6f75849b20231128bd3184bcc4744bf591f18e1d0b9b0eb7 of snapshot=snaptb-1701143704146 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-11-28 03:55:19,575 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c81e728d9d4c2f636f067f89cc14862c20231128548ceb87df594da68e7f4d2121a8bbf7 of snapshot=snaptb-1701143704146 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-11-28 03:55:19,585 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c81e728d9d4c2f636f067f89cc14862c20231128dec752e8fb644a0a80d6b4fd043402c1 of snapshot=snaptb-1701143704146 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-11-28 03:55:19,588 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c81e728d9d4c2f636f067f89cc14862c20231128eb89ca1113e144ddaad3b48ac062b47f of snapshot=snaptb-1701143704146 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-11-28 03:55:19,593 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c81e728d9d4c2f636f067f89cc14862c20231128fb09572884984543b4a9faae21b5679c of snapshot=snaptb-1701143704146 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-11-28 03:55:19,596 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink d41d8cd98f00b204e9800998ecf8427e202311280eef9bc4c106435986d8700c2b7868aa of snapshot=snaptb-1701143704146 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-11-28 03:55:19,600 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink d41d8cd98f00b204e9800998ecf8427e202311285c06394279144f2aa2ca5a78e6ddb299 of snapshot=snaptb-1701143704146 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-11-28 03:55:19,604 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink d41d8cd98f00b204e9800998ecf8427e2023112864ca880632ff4b81b856e56f97a51ba5 of snapshot=snaptb-1701143704146 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-11-28 03:55:19,608 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink d41d8cd98f00b204e9800998ecf8427e20231128ae21d695d78b47819031a63d23fd1adf of snapshot=snaptb-1701143704146 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-11-28 03:55:19,615 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b2023112847e6b2a73e5648f988213810b241c45a_del to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b2023112847e6b2a73e5648f988213810b241c45a_del 2023-11-28 03:55:19,616 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20231128f2a1b19354fb4a1ea87f959529de2a75_del to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20231128f2a1b19354fb4a1ea87f959529de2a75_del 2023-11-28 03:55:19,618 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20231128203e460f7da34a628c95173a08d56f3b_del to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20231128203e460f7da34a628c95173a08d56f3b_del 2023-11-28 03:55:19,618 INFO [PEWorker-3] snapshot.RestoreSnapshotHelper(223): region to restore: 2aad05dc9c73dcbea66259e5c0a6958f 2023-11-28 03:55:19,618 INFO [PEWorker-3] snapshot.RestoreSnapshotHelper(223): region to restore: 3d7f49de72f2e9e741f04714c6a7648c 2023-11-28 03:55:19,618 INFO [PEWorker-3] snapshot.RestoreSnapshotHelper(223): region to restore: b3c2fdb68f55a3cbe8d29ee7b6ddd5e5 2023-11-28 03:55:19,627 DEBUG [RestoreSnapshot-pool-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family1/b54918ba975e4d51b6b332b5e3e581de to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family1/b54918ba975e4d51b6b332b5e3e581de 2023-11-28 03:55:19,629 DEBUG [RestoreSnapshot-pool-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family1/caae163a11894964b761ab2b30894b6d_SeqId_29_ to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family1/caae163a11894964b761ab2b30894b6d_SeqId_29_ 2023-11-28 03:55:19,630 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 5bd4594355ca45ff98e17e8e298757d3 of snapshot=snaptb-1701143704146 to region=2aad05dc9c73dcbea66259e5c0a6958f table=testCompactionWithHFileLink 2023-11-28 03:55:19,630 DEBUG [RestoreSnapshot-pool-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family1/67c4337f7a424469ae26c6084ff7da42_SeqId_29_ to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family1/67c4337f7a424469ae26c6084ff7da42_SeqId_29_ 2023-11-28 03:55:19,631 DEBUG [RestoreSnapshot-pool-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family1/9c85e4a9998a42e797d2d126f730c0a6_SeqId_29_ to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family1/9c85e4a9998a42e797d2d126f730c0a6_SeqId_29_ 2023-11-28 03:55:19,632 DEBUG [RestoreSnapshot-pool-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family1/2c88d564dad043448d1c89f73401463e to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family1/2c88d564dad043448d1c89f73401463e 2023-11-28 03:55:19,632 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 88a76a3cbca146b6bb71214ecc4a9c87 of snapshot=snaptb-1701143704146 to region=b3c2fdb68f55a3cbe8d29ee7b6ddd5e5 table=testCompactionWithHFileLink 2023-11-28 03:55:19,633 DEBUG [RestoreSnapshot-pool-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family1/2c5387a00c534c1ba4b9b3b4a82a16db to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family1/2c5387a00c534c1ba4b9b3b4a82a16db 2023-11-28 03:55:19,633 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 9cba99270b974d6b9181045f6f7ee6bc of snapshot=snaptb-1701143704146 to region=3d7f49de72f2e9e741f04714c6a7648c table=testCompactionWithHFileLink 2023-11-28 03:55:19,642 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink ef8ca9f746b4468d8da6b892ab104927 of snapshot=snaptb-1701143704146 to region=2aad05dc9c73dcbea66259e5c0a6958f table=testCompactionWithHFileLink 2023-11-28 03:55:19,642 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 299d98e37656434ea5b2482137c58710 of snapshot=snaptb-1701143704146 to region=b3c2fdb68f55a3cbe8d29ee7b6ddd5e5 table=testCompactionWithHFileLink 2023-11-28 03:55:19,643 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 149c75fc3d8a444bbd7d2180c2614583 of snapshot=snaptb-1701143704146 to region=3d7f49de72f2e9e741f04714c6a7648c table=testCompactionWithHFileLink 2023-11-28 03:55:19,647 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 55b0944f49f248fd87ca616d0ed03e14 of snapshot=snaptb-1701143704146 to region=2aad05dc9c73dcbea66259e5c0a6958f table=testCompactionWithHFileLink 2023-11-28 03:55:19,647 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink b313b9e7ed474d04bf592f578c1928e8 of snapshot=snaptb-1701143704146 to region=3d7f49de72f2e9e741f04714c6a7648c table=testCompactionWithHFileLink 2023-11-28 03:55:19,648 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink dc7b41ef849648dda6e80bfb0cdeea0f of snapshot=snaptb-1701143704146 to region=b3c2fdb68f55a3cbe8d29ee7b6ddd5e5 table=testCompactionWithHFileLink 2023-11-28 03:55:19,650 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 0f49b13a3dc94f1bb9ed692753ce2e94 of snapshot=snaptb-1701143704146 to region=3d7f49de72f2e9e741f04714c6a7648c table=testCompactionWithHFileLink 2023-11-28 03:55:19,651 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink ab034fef3fd24871aeeb0dc81dfe66c2 of snapshot=snaptb-1701143704146 to region=2aad05dc9c73dcbea66259e5c0a6958f table=testCompactionWithHFileLink 2023-11-28 03:55:19,652 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 34bdbacde3fd4331ae0cc21041140fd9 of snapshot=snaptb-1701143704146 to region=b3c2fdb68f55a3cbe8d29ee7b6ddd5e5 table=testCompactionWithHFileLink 2023-11-28 03:55:19,656 DEBUG [RestoreSnapshot-pool-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family2/79ff143814eb45a58e52ea79560b8f93 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family2/79ff143814eb45a58e52ea79560b8f93 2023-11-28 03:55:19,656 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink cfbbdda0c47f440ab7343ce95b4baaf5 of snapshot=snaptb-1701143704146 to region=3d7f49de72f2e9e741f04714c6a7648c table=testCompactionWithHFileLink 2023-11-28 03:55:19,656 DEBUG [RestoreSnapshot-pool-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family2/b93f7d6b9dbf4058937d8aa40eb41bb5 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family2/b93f7d6b9dbf4058937d8aa40eb41bb5 2023-11-28 03:55:19,656 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink a79a53b7062d4e53b9199b02c23f6c24 of snapshot=snaptb-1701143704146 to region=2aad05dc9c73dcbea66259e5c0a6958f table=testCompactionWithHFileLink 2023-11-28 03:55:19,657 DEBUG [RestoreSnapshot-pool-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family2/c4c67664540a48089e64216c5442c7b5 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family2/c4c67664540a48089e64216c5442c7b5 2023-11-28 03:55:19,657 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink ee2d50b684b2479e8a02236cf5083faa of snapshot=snaptb-1701143704146 to region=b3c2fdb68f55a3cbe8d29ee7b6ddd5e5 table=testCompactionWithHFileLink 2023-11-28 03:55:19,659 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 14783dff7d074f04ac3290bc0e4a92ba of snapshot=snaptb-1701143704146 to region=3d7f49de72f2e9e741f04714c6a7648c table=testCompactionWithHFileLink 2023-11-28 03:55:19,660 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink a5d2b1b5f59d4745beda10e8ec8e729f of snapshot=snaptb-1701143704146 to region=2aad05dc9c73dcbea66259e5c0a6958f table=testCompactionWithHFileLink 2023-11-28 03:55:19,661 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 75ff1f4a28e9466e86a2f50fe4b3ba7c of snapshot=snaptb-1701143704146 to region=b3c2fdb68f55a3cbe8d29ee7b6ddd5e5 table=testCompactionWithHFileLink 2023-11-28 03:55:19,663 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink cf1816b4243e4ad197a1721e4235af5a of snapshot=snaptb-1701143704146 to region=3d7f49de72f2e9e741f04714c6a7648c table=testCompactionWithHFileLink 2023-11-28 03:55:19,664 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 9d803becce0f4ac4ad8171434ff9b0a2 of snapshot=snaptb-1701143704146 to region=2aad05dc9c73dcbea66259e5c0a6958f table=testCompactionWithHFileLink 2023-11-28 03:55:19,665 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink b06633c919b549ed9628414e3a0d7c29 of snapshot=snaptb-1701143704146 to region=b3c2fdb68f55a3cbe8d29ee7b6ddd5e5 table=testCompactionWithHFileLink 2023-11-28 03:55:19,667 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink a7b3c563e013482e86b9f3270fa9799c of snapshot=snaptb-1701143704146 to region=3d7f49de72f2e9e741f04714c6a7648c table=testCompactionWithHFileLink 2023-11-28 03:55:19,667 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink cdc6a50b83e147878684e9d41c01c3e6 of snapshot=snaptb-1701143704146 to region=2aad05dc9c73dcbea66259e5c0a6958f table=testCompactionWithHFileLink 2023-11-28 03:55:19,669 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink fe11679925e74afa9cd10ab81215b48a of snapshot=snaptb-1701143704146 to region=b3c2fdb68f55a3cbe8d29ee7b6ddd5e5 table=testCompactionWithHFileLink 2023-11-28 03:55:19,673 INFO [PEWorker-3] snapshot.RestoreSnapshotHelper(266): finishing restore table regions using snapshot=name: "snaptb-1701143704146" table: "testCompactionWithHFileLink" creation_time: 1701143704156 type: FLUSH version: 2 ttl: 0 2023-11-28 03:55:19,674 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testCompactionWithHFileLink,2,1701143695226.2aad05dc9c73dcbea66259e5c0a6958f.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1701143719674"}]},"ts":"9223372036854775807"} 2023-11-28 03:55:19,674 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testCompactionWithHFileLink,,1701143695226.3d7f49de72f2e9e741f04714c6a7648c.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1701143719674"}]},"ts":"9223372036854775807"} 2023-11-28 03:55:19,674 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testCompactionWithHFileLink,1,1701143695226.b3c2fdb68f55a3cbe8d29ee7b6ddd5e5.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1701143719674"}]},"ts":"9223372036854775807"} 2023-11-28 03:55:19,676 INFO [PEWorker-3] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-11-28 03:55:19,676 DEBUG [PEWorker-3] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 2aad05dc9c73dcbea66259e5c0a6958f, NAME => 'testCompactionWithHFileLink,2,1701143695226.2aad05dc9c73dcbea66259e5c0a6958f.', STARTKEY => '2', ENDKEY => ''}, {ENCODED => 3d7f49de72f2e9e741f04714c6a7648c, NAME => 'testCompactionWithHFileLink,,1701143695226.3d7f49de72f2e9e741f04714c6a7648c.', STARTKEY => '', ENDKEY => '1'}, {ENCODED => b3c2fdb68f55a3cbe8d29ee7b6ddd5e5, NAME => 'testCompactionWithHFileLink,1,1701143695226.b3c2fdb68f55a3cbe8d29ee7b6ddd5e5.', STARTKEY => '1', ENDKEY => '2'}] 2023-11-28 03:55:19,676 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,2,1701143695226.2aad05dc9c73dcbea66259e5c0a6958f.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1701143719675"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1701143719675"}]},"ts":"1701143719675"} 2023-11-28 03:55:19,676 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,,1701143695226.3d7f49de72f2e9e741f04714c6a7648c.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1701143719675"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1701143719675"}]},"ts":"1701143719675"} 2023-11-28 03:55:19,676 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,1,1701143695226.b3c2fdb68f55a3cbe8d29ee7b6ddd5e5.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1701143719675"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1701143719675"}]},"ts":"1701143719675"} 2023-11-28 03:55:19,678 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 3 regions to meta. 2023-11-28 03:55:19,678 INFO [PEWorker-3] hbase.MetaTableAccessor(1810): Overwritten 3 regions to Meta 2023-11-28 03:55:19,678 DEBUG [PEWorker-3] hbase.MetaTableAccessor(1811): Overwritten regions: [{ENCODED => 2aad05dc9c73dcbea66259e5c0a6958f, NAME => 'testCompactionWithHFileLink,2,1701143695226.2aad05dc9c73dcbea66259e5c0a6958f.', STARTKEY => '2', ENDKEY => ''}, {ENCODED => 3d7f49de72f2e9e741f04714c6a7648c, NAME => 'testCompactionWithHFileLink,,1701143695226.3d7f49de72f2e9e741f04714c6a7648c.', STARTKEY => '', ENDKEY => '1'}, {ENCODED => b3c2fdb68f55a3cbe8d29ee7b6ddd5e5, NAME => 'testCompactionWithHFileLink,1,1701143695226.b3c2fdb68f55a3cbe8d29ee7b6ddd5e5.', STARTKEY => '1', ENDKEY => '2'}] 2023-11-28 03:55:19,678 INFO [PEWorker-3] procedure.RestoreSnapshotProcedure(457): Restore snapshot={ ss=snaptb-1701143704146 table=testCompactionWithHFileLink type=FLUSH ttl=0 } on table=testCompactionWithHFileLink completed! 2023-11-28 03:55:19,680 INFO [PEWorker-3] procedure2.ProcedureExecutor(1409): Finished pid=96, state=SUCCESS; RestoreSnapshotProcedure (table=testCompactionWithHFileLink snapshot=name: "snaptb-1701143704146" table: "testCompactionWithHFileLink" creation_time: 1701143704156 type: FLUSH version: 2 ttl: 0 ) in 560 msec 2023-11-28 03:55:19,736 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] master.MasterRpcServices(1230): Checking to see if procedure is done pid=96 2023-11-28 03:55:19,736 INFO [Listener at localhost/42853] client.HBaseAdmin$TableFuture(3541): Operation: MODIFY, Table Name: default:testCompactionWithHFileLink, procId: 96 completed 2023-11-28 03:55:19,737 INFO [Listener at localhost/42853] client.HBaseAdmin(2618): Deleting restore-failsafe snapshot: hbase-failsafe-snaptb-1701143704146-1701143716957 2023-11-28 03:55:19,741 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] master.MasterRpcServices(751): Client=jenkins//172.31.14.131 delete name: "hbase-failsafe-snaptb-1701143704146-1701143716957" 2023-11-28 03:55:19,743 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] snapshot.SnapshotManager(335): Deleting snapshot: hbase-failsafe-snaptb-1701143704146-1701143716957 2023-11-28 03:55:19,745 INFO [Listener at localhost/42853] client.HBaseAdmin$14(830): Started enable of testCompactionWithHFileLink 2023-11-28 03:55:19,749 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] master.HMaster$10(2385): Client=jenkins//172.31.14.131 enable testCompactionWithHFileLink 2023-11-28 03:55:19,755 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] procedure2.ProcedureExecutor(1028): Stored pid=97, state=RUNNABLE:ENABLE_TABLE_PREPARE; EnableTableProcedure table=testCompactionWithHFileLink 2023-11-28 03:55:19,757 INFO [PEWorker-5] procedure.EnableTableProcedure(296): Attempting to enable the table testCompactionWithHFileLink 2023-11-28 03:55:19,757 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] master.MasterRpcServices(1230): Checking to see if procedure is done pid=97 2023-11-28 03:55:19,757 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1701143719757"}]},"ts":"1701143719757"} 2023-11-28 03:55:19,758 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=ENABLING in hbase:meta 2023-11-28 03:55:19,761 DEBUG [PEWorker-5] procedure.EnableTableProcedure(107): No change in number of region replicas (configuredReplicaCount=1); assigning. 2023-11-28 03:55:19,761 INFO [PEWorker-5] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=98, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=3d7f49de72f2e9e741f04714c6a7648c, ASSIGN}, {pid=99, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=b3c2fdb68f55a3cbe8d29ee7b6ddd5e5, ASSIGN}, {pid=100, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=2aad05dc9c73dcbea66259e5c0a6958f, ASSIGN}] 2023-11-28 03:55:19,762 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=100, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=2aad05dc9c73dcbea66259e5c0a6958f, ASSIGN 2023-11-28 03:55:19,762 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=99, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=b3c2fdb68f55a3cbe8d29ee7b6ddd5e5, ASSIGN 2023-11-28 03:55:19,762 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=98, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=3d7f49de72f2e9e741f04714c6a7648c, ASSIGN 2023-11-28 03:55:19,763 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=100, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=2aad05dc9c73dcbea66259e5c0a6958f, ASSIGN; state=CLOSED, location=null; forceNewPlan=false, retain=false 2023-11-28 03:55:19,763 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=99, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=b3c2fdb68f55a3cbe8d29ee7b6ddd5e5, ASSIGN; state=CLOSED, location=null; forceNewPlan=false, retain=false 2023-11-28 03:55:19,763 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=98, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=3d7f49de72f2e9e741f04714c6a7648c, ASSIGN; state=CLOSED, location=null; forceNewPlan=false, retain=false 2023-11-28 03:55:19,858 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] master.MasterRpcServices(1230): Checking to see if procedure is done pid=97 2023-11-28 03:55:19,914 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=100 updating hbase:meta row=2aad05dc9c73dcbea66259e5c0a6958f, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:19,914 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=98 updating hbase:meta row=3d7f49de72f2e9e741f04714c6a7648c, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:19,914 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=99 updating hbase:meta row=b3c2fdb68f55a3cbe8d29ee7b6ddd5e5, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:19,915 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,2,1701143695226.2aad05dc9c73dcbea66259e5c0a6958f.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1701143719914"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1701143719914"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1701143719914"}]},"ts":"1701143719914"} 2023-11-28 03:55:19,915 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,,1701143695226.3d7f49de72f2e9e741f04714c6a7648c.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1701143719914"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1701143719914"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1701143719914"}]},"ts":"1701143719914"} 2023-11-28 03:55:19,915 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,1,1701143695226.b3c2fdb68f55a3cbe8d29ee7b6ddd5e5.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1701143719914"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1701143719914"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1701143719914"}]},"ts":"1701143719914"} 2023-11-28 03:55:19,916 INFO [PEWorker-3] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=101, ppid=100, state=RUNNABLE; OpenRegionProcedure 2aad05dc9c73dcbea66259e5c0a6958f, server=jenkins-hbase4.apache.org,41203,1701143600653}] 2023-11-28 03:55:19,917 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=102, ppid=98, state=RUNNABLE; OpenRegionProcedure 3d7f49de72f2e9e741f04714c6a7648c, server=jenkins-hbase4.apache.org,41203,1701143600653}] 2023-11-28 03:55:19,918 INFO [PEWorker-5] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=103, ppid=99, state=RUNNABLE; OpenRegionProcedure b3c2fdb68f55a3cbe8d29ee7b6ddd5e5, server=jenkins-hbase4.apache.org,41203,1701143600653}] 2023-11-28 03:55:20,059 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] master.MasterRpcServices(1230): Checking to see if procedure is done pid=97 2023-11-28 03:55:20,069 DEBUG [RSProcedureDispatcher-pool-2] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:20,071 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testCompactionWithHFileLink,,1701143695226.3d7f49de72f2e9e741f04714c6a7648c. 2023-11-28 03:55:20,071 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 3d7f49de72f2e9e741f04714c6a7648c, NAME => 'testCompactionWithHFileLink,,1701143695226.3d7f49de72f2e9e741f04714c6a7648c.', STARTKEY => '', ENDKEY => '1'} 2023-11-28 03:55:20,072 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink 3d7f49de72f2e9e741f04714c6a7648c 2023-11-28 03:55:20,072 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,,1701143695226.3d7f49de72f2e9e741f04714c6a7648c.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-28 03:55:20,072 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 3d7f49de72f2e9e741f04714c6a7648c 2023-11-28 03:55:20,072 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 3d7f49de72f2e9e741f04714c6a7648c 2023-11-28 03:55:20,073 INFO [StoreOpener-3d7f49de72f2e9e741f04714c6a7648c-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 3d7f49de72f2e9e741f04714c6a7648c 2023-11-28 03:55:20,074 DEBUG [StoreOpener-3d7f49de72f2e9e741f04714c6a7648c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family1 2023-11-28 03:55:20,074 DEBUG [StoreOpener-3d7f49de72f2e9e741f04714c6a7648c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family1 2023-11-28 03:55:20,074 INFO [StoreOpener-3d7f49de72f2e9e741f04714c6a7648c-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 3d7f49de72f2e9e741f04714c6a7648c columnFamilyName family1 2023-11-28 03:55:20,085 DEBUG [StoreOpener-3d7f49de72f2e9e741f04714c6a7648c-1] regionserver.HStore(539): loaded hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family1/testCompactionWithHFileLink=3d7f49de72f2e9e741f04714c6a7648c-0f49b13a3dc94f1bb9ed692753ce2e94 2023-11-28 03:55:20,091 DEBUG [StoreOpener-3d7f49de72f2e9e741f04714c6a7648c-1] regionserver.HStore(539): loaded hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family1/testCompactionWithHFileLink=3d7f49de72f2e9e741f04714c6a7648c-149c75fc3d8a444bbd7d2180c2614583 2023-11-28 03:55:20,096 DEBUG [StoreOpener-3d7f49de72f2e9e741f04714c6a7648c-1] regionserver.HStore(539): loaded hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family1/testCompactionWithHFileLink=3d7f49de72f2e9e741f04714c6a7648c-9cba99270b974d6b9181045f6f7ee6bc 2023-11-28 03:55:20,102 DEBUG [StoreOpener-3d7f49de72f2e9e741f04714c6a7648c-1] regionserver.HStore(539): loaded hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family1/testCompactionWithHFileLink=3d7f49de72f2e9e741f04714c6a7648c-b313b9e7ed474d04bf592f578c1928e8 2023-11-28 03:55:20,102 INFO [StoreOpener-3d7f49de72f2e9e741f04714c6a7648c-1] regionserver.HStore(310): Store=3d7f49de72f2e9e741f04714c6a7648c/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-28 03:55:20,102 INFO [StoreOpener-3d7f49de72f2e9e741f04714c6a7648c-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 3d7f49de72f2e9e741f04714c6a7648c 2023-11-28 03:55:20,103 DEBUG [StoreOpener-3d7f49de72f2e9e741f04714c6a7648c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family2 2023-11-28 03:55:20,103 DEBUG [StoreOpener-3d7f49de72f2e9e741f04714c6a7648c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family2 2023-11-28 03:55:20,104 INFO [StoreOpener-3d7f49de72f2e9e741f04714c6a7648c-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 3d7f49de72f2e9e741f04714c6a7648c columnFamilyName family2 2023-11-28 03:55:20,111 DEBUG [StoreOpener-3d7f49de72f2e9e741f04714c6a7648c-1] regionserver.HStore(539): loaded hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family2/testCompactionWithHFileLink=3d7f49de72f2e9e741f04714c6a7648c-14783dff7d074f04ac3290bc0e4a92ba 2023-11-28 03:55:20,118 DEBUG [StoreOpener-3d7f49de72f2e9e741f04714c6a7648c-1] regionserver.HStore(539): loaded hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family2/testCompactionWithHFileLink=3d7f49de72f2e9e741f04714c6a7648c-a7b3c563e013482e86b9f3270fa9799c 2023-11-28 03:55:20,123 DEBUG [StoreOpener-3d7f49de72f2e9e741f04714c6a7648c-1] regionserver.HStore(539): loaded hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family2/testCompactionWithHFileLink=3d7f49de72f2e9e741f04714c6a7648c-cf1816b4243e4ad197a1721e4235af5a 2023-11-28 03:55:20,129 DEBUG [StoreOpener-3d7f49de72f2e9e741f04714c6a7648c-1] regionserver.HStore(539): loaded hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family2/testCompactionWithHFileLink=3d7f49de72f2e9e741f04714c6a7648c-cfbbdda0c47f440ab7343ce95b4baaf5 2023-11-28 03:55:20,129 INFO [StoreOpener-3d7f49de72f2e9e741f04714c6a7648c-1] regionserver.HStore(310): Store=3d7f49de72f2e9e741f04714c6a7648c/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-28 03:55:20,131 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c 2023-11-28 03:55:20,132 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c 2023-11-28 03:55:20,134 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-11-28 03:55:20,135 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 3d7f49de72f2e9e741f04714c6a7648c 2023-11-28 03:55:20,136 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 3d7f49de72f2e9e741f04714c6a7648c; next sequenceid=32; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63119240, jitterRate=-0.05945003032684326}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-11-28 03:55:20,136 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 3d7f49de72f2e9e741f04714c6a7648c: 2023-11-28 03:55:20,137 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2339): Post open deploy tasks for testCompactionWithHFileLink,,1701143695226.3d7f49de72f2e9e741f04714c6a7648c., pid=102, masterSystemTime=1701143720068 2023-11-28 03:55:20,137 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-28 03:55:20,137 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-11-28 03:55:20,140 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-11-28 03:55:20,140 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2366): Finished post open deploy task for testCompactionWithHFileLink,,1701143695226.3d7f49de72f2e9e741f04714c6a7648c. 2023-11-28 03:55:20,140 DEBUG [RS:0;jenkins-hbase4:41203-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-11-28 03:55:20,140 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testCompactionWithHFileLink,,1701143695226.3d7f49de72f2e9e741f04714c6a7648c. 2023-11-28 03:55:20,140 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testCompactionWithHFileLink,2,1701143695226.2aad05dc9c73dcbea66259e5c0a6958f. 2023-11-28 03:55:20,140 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 2aad05dc9c73dcbea66259e5c0a6958f, NAME => 'testCompactionWithHFileLink,2,1701143695226.2aad05dc9c73dcbea66259e5c0a6958f.', STARTKEY => '2', ENDKEY => ''} 2023-11-28 03:55:20,140 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=98 updating hbase:meta row=3d7f49de72f2e9e741f04714c6a7648c, regionState=OPEN, openSeqNum=32, regionLocation=jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:20,141 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,,1701143695226.3d7f49de72f2e9e741f04714c6a7648c.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1701143720140"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1701143720140"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1701143720140"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1701143720140"}]},"ts":"1701143720140"} 2023-11-28 03:55:20,141 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink 2aad05dc9c73dcbea66259e5c0a6958f 2023-11-28 03:55:20,141 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,2,1701143695226.2aad05dc9c73dcbea66259e5c0a6958f.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-28 03:55:20,141 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 2aad05dc9c73dcbea66259e5c0a6958f 2023-11-28 03:55:20,141 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 2aad05dc9c73dcbea66259e5c0a6958f 2023-11-28 03:55:20,144 INFO [PEWorker-1] procedure2.ProcedureExecutor(1825): Finished subprocedure pid=102, resume processing ppid=98 2023-11-28 03:55:20,144 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=102, ppid=98, state=SUCCESS; OpenRegionProcedure 3d7f49de72f2e9e741f04714c6a7648c, server=jenkins-hbase4.apache.org,41203,1701143600653 in 225 msec 2023-11-28 03:55:20,145 INFO [PEWorker-4] procedure2.ProcedureExecutor(1409): Finished pid=98, ppid=97, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=3d7f49de72f2e9e741f04714c6a7648c, ASSIGN in 383 msec 2023-11-28 03:55:20,147 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 4 files of size 22152 starting at candidate #0 after considering 3 permutations with 3 in ratio 2023-11-28 03:55:20,147 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] regionserver.HStore(1912): 3d7f49de72f2e9e741f04714c6a7648c/family1 is initiating minor compaction (all files) 2023-11-28 03:55:20,147 INFO [RS:0;jenkins-hbase4:41203-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 3d7f49de72f2e9e741f04714c6a7648c/family1 in testCompactionWithHFileLink,,1701143695226.3d7f49de72f2e9e741f04714c6a7648c. 2023-11-28 03:55:20,147 INFO [RS:0;jenkins-hbase4:41203-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family1/testCompactionWithHFileLink=3d7f49de72f2e9e741f04714c6a7648c-9cba99270b974d6b9181045f6f7ee6bc, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family1/testCompactionWithHFileLink=3d7f49de72f2e9e741f04714c6a7648c-149c75fc3d8a444bbd7d2180c2614583, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family1/testCompactionWithHFileLink=3d7f49de72f2e9e741f04714c6a7648c-b313b9e7ed474d04bf592f578c1928e8, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family1/testCompactionWithHFileLink=3d7f49de72f2e9e741f04714c6a7648c-0f49b13a3dc94f1bb9ed692753ce2e94] into tmpdir=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/.tmp, totalSize=21.6 K 2023-11-28 03:55:20,149 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=3d7f49de72f2e9e741f04714c6a7648c-9cba99270b974d6b9181045f6f7ee6bc, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1701143697342 2023-11-28 03:55:20,149 DEBUG [RS:0;jenkins-hbase4:41203-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 4 files of size 20920 starting at candidate #0 after considering 3 permutations with 3 in ratio 2023-11-28 03:55:20,149 DEBUG [RS:0;jenkins-hbase4:41203-longCompactions-0] regionserver.HStore(1912): 3d7f49de72f2e9e741f04714c6a7648c/family2 is initiating minor compaction (all files) 2023-11-28 03:55:20,149 INFO [RS:0;jenkins-hbase4:41203-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 3d7f49de72f2e9e741f04714c6a7648c/family2 in testCompactionWithHFileLink,,1701143695226.3d7f49de72f2e9e741f04714c6a7648c. 2023-11-28 03:55:20,149 INFO [RS:0;jenkins-hbase4:41203-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family2/testCompactionWithHFileLink=3d7f49de72f2e9e741f04714c6a7648c-cfbbdda0c47f440ab7343ce95b4baaf5, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family2/testCompactionWithHFileLink=3d7f49de72f2e9e741f04714c6a7648c-14783dff7d074f04ac3290bc0e4a92ba, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family2/testCompactionWithHFileLink=3d7f49de72f2e9e741f04714c6a7648c-cf1816b4243e4ad197a1721e4235af5a, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family2/testCompactionWithHFileLink=3d7f49de72f2e9e741f04714c6a7648c-a7b3c563e013482e86b9f3270fa9799c] into tmpdir=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/.tmp, totalSize=20.4 K 2023-11-28 03:55:20,149 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=3d7f49de72f2e9e741f04714c6a7648c-149c75fc3d8a444bbd7d2180c2614583, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1701143698738 2023-11-28 03:55:20,150 DEBUG [RS:0;jenkins-hbase4:41203-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=3d7f49de72f2e9e741f04714c6a7648c-cfbbdda0c47f440ab7343ce95b4baaf5, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1701143697342 2023-11-28 03:55:20,150 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=3d7f49de72f2e9e741f04714c6a7648c-b313b9e7ed474d04bf592f578c1928e8, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1701143700547 2023-11-28 03:55:20,151 DEBUG [RS:0;jenkins-hbase4:41203-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=3d7f49de72f2e9e741f04714c6a7648c-14783dff7d074f04ac3290bc0e4a92ba, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1701143698738 2023-11-28 03:55:20,151 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=3d7f49de72f2e9e741f04714c6a7648c-0f49b13a3dc94f1bb9ed692753ce2e94, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1701143702350 2023-11-28 03:55:20,152 DEBUG [RS:0;jenkins-hbase4:41203-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=3d7f49de72f2e9e741f04714c6a7648c-cf1816b4243e4ad197a1721e4235af5a, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1701143700547 2023-11-28 03:55:20,152 DEBUG [RS:0;jenkins-hbase4:41203-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=3d7f49de72f2e9e741f04714c6a7648c-a7b3c563e013482e86b9f3270fa9799c, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1701143702350 2023-11-28 03:55:20,158 INFO [StoreOpener-2aad05dc9c73dcbea66259e5c0a6958f-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 2aad05dc9c73dcbea66259e5c0a6958f 2023-11-28 03:55:20,160 DEBUG [StoreOpener-2aad05dc9c73dcbea66259e5c0a6958f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family1 2023-11-28 03:55:20,161 DEBUG [StoreOpener-2aad05dc9c73dcbea66259e5c0a6958f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family1 2023-11-28 03:55:20,161 INFO [StoreOpener-2aad05dc9c73dcbea66259e5c0a6958f-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 2aad05dc9c73dcbea66259e5c0a6958f columnFamilyName family1 2023-11-28 03:55:20,198 DEBUG [StoreOpener-2aad05dc9c73dcbea66259e5c0a6958f-1] regionserver.HStore(539): loaded hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family1/testCompactionWithHFileLink=2aad05dc9c73dcbea66259e5c0a6958f-55b0944f49f248fd87ca616d0ed03e14 2023-11-28 03:55:20,215 DEBUG [StoreOpener-2aad05dc9c73dcbea66259e5c0a6958f-1] regionserver.HStore(539): loaded hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family1/testCompactionWithHFileLink=2aad05dc9c73dcbea66259e5c0a6958f-5bd4594355ca45ff98e17e8e298757d3 2023-11-28 03:55:20,220 INFO [RS:0;jenkins-hbase4:41203-longCompactions-0] throttle.PressureAwareThroughputController(145): 3d7f49de72f2e9e741f04714c6a7648c#family2#compaction#113 average throughput is 0.19 MB/second, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-11-28 03:55:20,225 INFO [RS:0;jenkins-hbase4:41203-shortCompactions-0] throttle.PressureAwareThroughputController(145): 3d7f49de72f2e9e741f04714c6a7648c#family1#compaction#114 average throughput is 0.21 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-11-28 03:55:20,241 DEBUG [StoreOpener-2aad05dc9c73dcbea66259e5c0a6958f-1] regionserver.HStore(539): loaded hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family1/testCompactionWithHFileLink=2aad05dc9c73dcbea66259e5c0a6958f-ab034fef3fd24871aeeb0dc81dfe66c2 2023-11-28 03:55:20,249 WARN [DataStreamer for file /user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202311283573ab9fe57844b39c2ad33f3a65d42b_del block BP-1757459218-172.31.14.131-1701143595945:blk_1073742156_1332] hdfs.DataStreamer(823): DataStreamer Exception java.nio.channels.ClosedByInterruptException at java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:202) at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:477) at org.apache.hadoop.net.SocketOutputStream$Writer.performIO(SocketOutputStream.java:63) at org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:142) at org.apache.hadoop.net.SocketOutputStream.write(SocketOutputStream.java:159) at org.apache.hadoop.net.SocketOutputStream.write(SocketOutputStream.java:117) at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82) at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140) at java.io.DataOutputStream.flush(DataOutputStream.java:123) at org.apache.hadoop.hdfs.DataStreamer.run(DataStreamer.java:772) 2023-11-28 03:55:20,257 DEBUG [StoreOpener-2aad05dc9c73dcbea66259e5c0a6958f-1] regionserver.HStore(539): loaded hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family1/testCompactionWithHFileLink=2aad05dc9c73dcbea66259e5c0a6958f-ef8ca9f746b4468d8da6b892ab104927 2023-11-28 03:55:20,260 INFO [StoreOpener-2aad05dc9c73dcbea66259e5c0a6958f-1] regionserver.HStore(310): Store=2aad05dc9c73dcbea66259e5c0a6958f/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-28 03:55:20,261 INFO [StoreOpener-2aad05dc9c73dcbea66259e5c0a6958f-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 2aad05dc9c73dcbea66259e5c0a6958f 2023-11-28 03:55:20,262 DEBUG [StoreOpener-2aad05dc9c73dcbea66259e5c0a6958f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family2 2023-11-28 03:55:20,262 DEBUG [StoreOpener-2aad05dc9c73dcbea66259e5c0a6958f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family2 2023-11-28 03:55:20,262 INFO [StoreOpener-2aad05dc9c73dcbea66259e5c0a6958f-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 2aad05dc9c73dcbea66259e5c0a6958f columnFamilyName family2 2023-11-28 03:55:20,283 DEBUG [StoreOpener-2aad05dc9c73dcbea66259e5c0a6958f-1] regionserver.HStore(539): loaded hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family2/testCompactionWithHFileLink=2aad05dc9c73dcbea66259e5c0a6958f-9d803becce0f4ac4ad8171434ff9b0a2 2023-11-28 03:55:20,288 DEBUG [StoreOpener-2aad05dc9c73dcbea66259e5c0a6958f-1] regionserver.HStore(539): loaded hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family2/testCompactionWithHFileLink=2aad05dc9c73dcbea66259e5c0a6958f-a5d2b1b5f59d4745beda10e8ec8e729f 2023-11-28 03:55:20,295 DEBUG [StoreOpener-2aad05dc9c73dcbea66259e5c0a6958f-1] regionserver.HStore(539): loaded hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family2/testCompactionWithHFileLink=2aad05dc9c73dcbea66259e5c0a6958f-a79a53b7062d4e53b9199b02c23f6c24 2023-11-28 03:55:20,300 DEBUG [StoreOpener-2aad05dc9c73dcbea66259e5c0a6958f-1] regionserver.HStore(539): loaded hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family2/testCompactionWithHFileLink=2aad05dc9c73dcbea66259e5c0a6958f-cdc6a50b83e147878684e9d41c01c3e6 2023-11-28 03:55:20,300 INFO [StoreOpener-2aad05dc9c73dcbea66259e5c0a6958f-1] regionserver.HStore(310): Store=2aad05dc9c73dcbea66259e5c0a6958f/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-28 03:55:20,301 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f 2023-11-28 03:55:20,302 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f 2023-11-28 03:55:20,303 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-11-28 03:55:20,304 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 2aad05dc9c73dcbea66259e5c0a6958f 2023-11-28 03:55:20,305 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 2aad05dc9c73dcbea66259e5c0a6958f; next sequenceid=32; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=70564613, jitterRate=0.051494672894477844}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-11-28 03:55:20,305 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 2aad05dc9c73dcbea66259e5c0a6958f: 2023-11-28 03:55:20,306 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2339): Post open deploy tasks for testCompactionWithHFileLink,2,1701143695226.2aad05dc9c73dcbea66259e5c0a6958f., pid=101, masterSystemTime=1701143720068 2023-11-28 03:55:20,306 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-11-28 03:55:20,306 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=2), splitQueue=0 2023-11-28 03:55:20,307 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2366): Finished post open deploy task for testCompactionWithHFileLink,2,1701143695226.2aad05dc9c73dcbea66259e5c0a6958f. 2023-11-28 03:55:20,307 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testCompactionWithHFileLink,2,1701143695226.2aad05dc9c73dcbea66259e5c0a6958f. 2023-11-28 03:55:20,307 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testCompactionWithHFileLink,1,1701143695226.b3c2fdb68f55a3cbe8d29ee7b6ddd5e5. 2023-11-28 03:55:20,307 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => b3c2fdb68f55a3cbe8d29ee7b6ddd5e5, NAME => 'testCompactionWithHFileLink,1,1701143695226.b3c2fdb68f55a3cbe8d29ee7b6ddd5e5.', STARTKEY => '1', ENDKEY => '2'} 2023-11-28 03:55:20,308 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink b3c2fdb68f55a3cbe8d29ee7b6ddd5e5 2023-11-28 03:55:20,308 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=100 updating hbase:meta row=2aad05dc9c73dcbea66259e5c0a6958f, regionState=OPEN, openSeqNum=32, regionLocation=jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:20,308 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,1,1701143695226.b3c2fdb68f55a3cbe8d29ee7b6ddd5e5.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-28 03:55:20,308 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for b3c2fdb68f55a3cbe8d29ee7b6ddd5e5 2023-11-28 03:55:20,308 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for b3c2fdb68f55a3cbe8d29ee7b6ddd5e5 2023-11-28 03:55:20,308 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,2,1701143695226.2aad05dc9c73dcbea66259e5c0a6958f.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1701143720307"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1701143720307"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1701143720307"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1701143720307"}]},"ts":"1701143720307"} 2023-11-28 03:55:20,311 INFO [PEWorker-5] procedure2.ProcedureExecutor(1825): Finished subprocedure pid=101, resume processing ppid=100 2023-11-28 03:55:20,311 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=101, ppid=100, state=SUCCESS; OpenRegionProcedure 2aad05dc9c73dcbea66259e5c0a6958f, server=jenkins-hbase4.apache.org,41203,1701143600653 in 393 msec 2023-11-28 03:55:20,311 INFO [StoreOpener-b3c2fdb68f55a3cbe8d29ee7b6ddd5e5-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region b3c2fdb68f55a3cbe8d29ee7b6ddd5e5 2023-11-28 03:55:20,312 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=100, ppid=97, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=2aad05dc9c73dcbea66259e5c0a6958f, ASSIGN in 550 msec 2023-11-28 03:55:20,312 DEBUG [StoreOpener-b3c2fdb68f55a3cbe8d29ee7b6ddd5e5-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family1 2023-11-28 03:55:20,312 DEBUG [StoreOpener-b3c2fdb68f55a3cbe8d29ee7b6ddd5e5-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family1 2023-11-28 03:55:20,313 INFO [StoreOpener-b3c2fdb68f55a3cbe8d29ee7b6ddd5e5-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region b3c2fdb68f55a3cbe8d29ee7b6ddd5e5 columnFamilyName family1 2023-11-28 03:55:20,323 DEBUG [StoreOpener-b3c2fdb68f55a3cbe8d29ee7b6ddd5e5-1] regionserver.HStore(539): loaded hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family1/testCompactionWithHFileLink=b3c2fdb68f55a3cbe8d29ee7b6ddd5e5-299d98e37656434ea5b2482137c58710 2023-11-28 03:55:20,327 DEBUG [StoreOpener-b3c2fdb68f55a3cbe8d29ee7b6ddd5e5-1] regionserver.HStore(539): loaded hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family1/testCompactionWithHFileLink=b3c2fdb68f55a3cbe8d29ee7b6ddd5e5-34bdbacde3fd4331ae0cc21041140fd9 2023-11-28 03:55:20,334 DEBUG [StoreOpener-b3c2fdb68f55a3cbe8d29ee7b6ddd5e5-1] regionserver.HStore(539): loaded hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family1/testCompactionWithHFileLink=b3c2fdb68f55a3cbe8d29ee7b6ddd5e5-88a76a3cbca146b6bb71214ecc4a9c87 2023-11-28 03:55:20,339 DEBUG [StoreOpener-b3c2fdb68f55a3cbe8d29ee7b6ddd5e5-1] regionserver.HStore(539): loaded hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family1/testCompactionWithHFileLink=b3c2fdb68f55a3cbe8d29ee7b6ddd5e5-dc7b41ef849648dda6e80bfb0cdeea0f 2023-11-28 03:55:20,339 INFO [StoreOpener-b3c2fdb68f55a3cbe8d29ee7b6ddd5e5-1] regionserver.HStore(310): Store=b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-28 03:55:20,339 INFO [StoreOpener-b3c2fdb68f55a3cbe8d29ee7b6ddd5e5-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region b3c2fdb68f55a3cbe8d29ee7b6ddd5e5 2023-11-28 03:55:20,340 DEBUG [StoreOpener-b3c2fdb68f55a3cbe8d29ee7b6ddd5e5-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family2 2023-11-28 03:55:20,340 DEBUG [StoreOpener-b3c2fdb68f55a3cbe8d29ee7b6ddd5e5-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family2 2023-11-28 03:55:20,341 INFO [StoreOpener-b3c2fdb68f55a3cbe8d29ee7b6ddd5e5-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region b3c2fdb68f55a3cbe8d29ee7b6ddd5e5 columnFamilyName family2 2023-11-28 03:55:20,348 DEBUG [StoreOpener-b3c2fdb68f55a3cbe8d29ee7b6ddd5e5-1] regionserver.HStore(539): loaded hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family2/testCompactionWithHFileLink=b3c2fdb68f55a3cbe8d29ee7b6ddd5e5-75ff1f4a28e9466e86a2f50fe4b3ba7c 2023-11-28 03:55:20,354 DEBUG [StoreOpener-b3c2fdb68f55a3cbe8d29ee7b6ddd5e5-1] regionserver.HStore(539): loaded hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family2/testCompactionWithHFileLink=b3c2fdb68f55a3cbe8d29ee7b6ddd5e5-b06633c919b549ed9628414e3a0d7c29 2023-11-28 03:55:20,360 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] master.MasterRpcServices(1230): Checking to see if procedure is done pid=97 2023-11-28 03:55:20,361 DEBUG [StoreOpener-b3c2fdb68f55a3cbe8d29ee7b6ddd5e5-1] regionserver.HStore(539): loaded hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family2/testCompactionWithHFileLink=b3c2fdb68f55a3cbe8d29ee7b6ddd5e5-ee2d50b684b2479e8a02236cf5083faa 2023-11-28 03:55:20,366 DEBUG [StoreOpener-b3c2fdb68f55a3cbe8d29ee7b6ddd5e5-1] regionserver.HStore(539): loaded hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family2/testCompactionWithHFileLink=b3c2fdb68f55a3cbe8d29ee7b6ddd5e5-fe11679925e74afa9cd10ab81215b48a 2023-11-28 03:55:20,367 INFO [StoreOpener-b3c2fdb68f55a3cbe8d29ee7b6ddd5e5-1] regionserver.HStore(310): Store=b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-28 03:55:20,370 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5 2023-11-28 03:55:20,372 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5 2023-11-28 03:55:20,375 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-11-28 03:55:20,376 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for b3c2fdb68f55a3cbe8d29ee7b6ddd5e5 2023-11-28 03:55:20,377 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened b3c2fdb68f55a3cbe8d29ee7b6ddd5e5; next sequenceid=32; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=73913984, jitterRate=0.10140419006347656}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-11-28 03:55:20,377 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for b3c2fdb68f55a3cbe8d29ee7b6ddd5e5: 2023-11-28 03:55:20,378 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2339): Post open deploy tasks for testCompactionWithHFileLink,1,1701143695226.b3c2fdb68f55a3cbe8d29ee7b6ddd5e5., pid=103, masterSystemTime=1701143720068 2023-11-28 03:55:20,378 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=3), splitQueue=0 2023-11-28 03:55:20,378 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=4), splitQueue=0 2023-11-28 03:55:20,380 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2366): Finished post open deploy task for testCompactionWithHFileLink,1,1701143695226.b3c2fdb68f55a3cbe8d29ee7b6ddd5e5. 2023-11-28 03:55:20,380 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testCompactionWithHFileLink,1,1701143695226.b3c2fdb68f55a3cbe8d29ee7b6ddd5e5. 2023-11-28 03:55:20,380 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=99 updating hbase:meta row=b3c2fdb68f55a3cbe8d29ee7b6ddd5e5, regionState=OPEN, openSeqNum=32, regionLocation=jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:20,380 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,1,1701143695226.b3c2fdb68f55a3cbe8d29ee7b6ddd5e5.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1701143720380"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1701143720380"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1701143720380"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1701143720380"}]},"ts":"1701143720380"} 2023-11-28 03:55:20,383 INFO [PEWorker-3] procedure2.ProcedureExecutor(1825): Finished subprocedure pid=103, resume processing ppid=99 2023-11-28 03:55:20,383 INFO [PEWorker-3] procedure2.ProcedureExecutor(1409): Finished pid=103, ppid=99, state=SUCCESS; OpenRegionProcedure b3c2fdb68f55a3cbe8d29ee7b6ddd5e5, server=jenkins-hbase4.apache.org,41203,1701143600653 in 464 msec 2023-11-28 03:55:20,385 INFO [PEWorker-1] procedure2.ProcedureExecutor(1825): Finished subprocedure pid=99, resume processing ppid=97 2023-11-28 03:55:20,385 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=99, ppid=97, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=b3c2fdb68f55a3cbe8d29ee7b6ddd5e5, ASSIGN in 622 msec 2023-11-28 03:55:20,385 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1701143720385"}]},"ts":"1701143720385"} 2023-11-28 03:55:20,386 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=ENABLED in hbase:meta 2023-11-28 03:55:20,389 INFO [PEWorker-4] procedure.EnableTableProcedure(310): Table 'testCompactionWithHFileLink' was successfully enabled. 2023-11-28 03:55:20,391 INFO [PEWorker-4] procedure2.ProcedureExecutor(1409): Finished pid=97, state=SUCCESS; EnableTableProcedure table=testCompactionWithHFileLink in 640 msec 2023-11-28 03:55:20,696 DEBUG [RS:0;jenkins-hbase4:41203-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/.tmp/family2/5592e91e22fd4110851c435d5e013d74 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family2/5592e91e22fd4110851c435d5e013d74 2023-11-28 03:55:20,705 INFO [RS:0;jenkins-hbase4:41203-longCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in 3d7f49de72f2e9e741f04714c6a7648c/family2 of 3d7f49de72f2e9e741f04714c6a7648c into 5592e91e22fd4110851c435d5e013d74(size=6.4 K), total size for store is 6.4 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-11-28 03:55:20,705 DEBUG [RS:0;jenkins-hbase4:41203-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 3d7f49de72f2e9e741f04714c6a7648c: 2023-11-28 03:55:20,705 INFO [RS:0;jenkins-hbase4:41203-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testCompactionWithHFileLink,,1701143695226.3d7f49de72f2e9e741f04714c6a7648c., storeName=3d7f49de72f2e9e741f04714c6a7648c/family2, priority=12, startTime=1701143720137; duration=0sec 2023-11-28 03:55:20,705 DEBUG [RS:0;jenkins-hbase4:41203-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=4), splitQueue=0 2023-11-28 03:55:20,706 DEBUG [RS:0;jenkins-hbase4:41203-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-11-28 03:55:20,709 DEBUG [RS:0;jenkins-hbase4:41203-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 4 files of size 22152 starting at candidate #0 after considering 3 permutations with 3 in ratio 2023-11-28 03:55:20,709 DEBUG [RS:0;jenkins-hbase4:41203-longCompactions-0] regionserver.HStore(1912): 2aad05dc9c73dcbea66259e5c0a6958f/family1 is initiating minor compaction (all files) 2023-11-28 03:55:20,709 INFO [RS:0;jenkins-hbase4:41203-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 2aad05dc9c73dcbea66259e5c0a6958f/family1 in testCompactionWithHFileLink,2,1701143695226.2aad05dc9c73dcbea66259e5c0a6958f. 2023-11-28 03:55:20,709 INFO [RS:0;jenkins-hbase4:41203-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family1/testCompactionWithHFileLink=2aad05dc9c73dcbea66259e5c0a6958f-5bd4594355ca45ff98e17e8e298757d3, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family1/testCompactionWithHFileLink=2aad05dc9c73dcbea66259e5c0a6958f-ef8ca9f746b4468d8da6b892ab104927, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family1/testCompactionWithHFileLink=2aad05dc9c73dcbea66259e5c0a6958f-55b0944f49f248fd87ca616d0ed03e14, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family1/testCompactionWithHFileLink=2aad05dc9c73dcbea66259e5c0a6958f-ab034fef3fd24871aeeb0dc81dfe66c2] into tmpdir=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/.tmp, totalSize=21.6 K 2023-11-28 03:55:20,710 DEBUG [RS:0;jenkins-hbase4:41203-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=2aad05dc9c73dcbea66259e5c0a6958f-5bd4594355ca45ff98e17e8e298757d3, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1701143697344 2023-11-28 03:55:20,711 DEBUG [RS:0;jenkins-hbase4:41203-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=2aad05dc9c73dcbea66259e5c0a6958f-ef8ca9f746b4468d8da6b892ab104927, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1701143698739 2023-11-28 03:55:20,711 DEBUG [RS:0;jenkins-hbase4:41203-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=2aad05dc9c73dcbea66259e5c0a6958f-55b0944f49f248fd87ca616d0ed03e14, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1701143700547 2023-11-28 03:55:20,712 DEBUG [RS:0;jenkins-hbase4:41203-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=2aad05dc9c73dcbea66259e5c0a6958f-ab034fef3fd24871aeeb0dc81dfe66c2, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1701143702350 2023-11-28 03:55:20,736 INFO [RS:0;jenkins-hbase4:41203-longCompactions-0] throttle.PressureAwareThroughputController(145): 2aad05dc9c73dcbea66259e5c0a6958f#family1#compaction#115 average throughput is 0.78 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-11-28 03:55:20,805 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family2/testCompactionWithHFileLink=3d7f49de72f2e9e741f04714c6a7648c-cfbbdda0c47f440ab7343ce95b4baaf5, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family2/testCompactionWithHFileLink=3d7f49de72f2e9e741f04714c6a7648c-14783dff7d074f04ac3290bc0e4a92ba, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family2/testCompactionWithHFileLink=3d7f49de72f2e9e741f04714c6a7648c-cf1816b4243e4ad197a1721e4235af5a, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family2/testCompactionWithHFileLink=3d7f49de72f2e9e741f04714c6a7648c-a7b3c563e013482e86b9f3270fa9799c] to archive 2023-11-28 03:55:20,806 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-28 03:55:20,808 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family2/testCompactionWithHFileLink=3d7f49de72f2e9e741f04714c6a7648c-cfbbdda0c47f440ab7343ce95b4baaf5 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family2/testCompactionWithHFileLink=3d7f49de72f2e9e741f04714c6a7648c-cfbbdda0c47f440ab7343ce95b4baaf5 2023-11-28 03:55:20,810 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family2/testCompactionWithHFileLink=3d7f49de72f2e9e741f04714c6a7648c-14783dff7d074f04ac3290bc0e4a92ba to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family2/testCompactionWithHFileLink=3d7f49de72f2e9e741f04714c6a7648c-14783dff7d074f04ac3290bc0e4a92ba 2023-11-28 03:55:20,811 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family2/testCompactionWithHFileLink=3d7f49de72f2e9e741f04714c6a7648c-cf1816b4243e4ad197a1721e4235af5a to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family2/testCompactionWithHFileLink=3d7f49de72f2e9e741f04714c6a7648c-cf1816b4243e4ad197a1721e4235af5a 2023-11-28 03:55:20,813 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family2/testCompactionWithHFileLink=3d7f49de72f2e9e741f04714c6a7648c-a7b3c563e013482e86b9f3270fa9799c to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family2/testCompactionWithHFileLink=3d7f49de72f2e9e741f04714c6a7648c-a7b3c563e013482e86b9f3270fa9799c 2023-11-28 03:55:20,861 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] master.MasterRpcServices(1230): Checking to see if procedure is done pid=97 2023-11-28 03:55:20,862 INFO [Listener at localhost/42853] client.HBaseAdmin$TableFuture(3541): Operation: ENABLE, Table Name: default:testCompactionWithHFileLink, procId: 97 completed 2023-11-28 03:55:20,871 DEBUG [Listener at localhost/42853] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-11-28 03:55:20,879 DEBUG [Listener at localhost/42853] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-11-28 03:55:20,888 DEBUG [Listener at localhost/42853] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-11-28 03:55:20,889 DEBUG [Listener at localhost/42853] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-11-28 03:55:20,894 INFO [Listener at localhost/42853] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-11-28 03:55:20,907 INFO [Listener at localhost/42853] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 12 selected files, and 0 irrelevant files table 'testCompactionWithHFileLink' and column 'family1' 2023-11-28 03:55:20,908 INFO [Listener at localhost/42853] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testCompactionWithHFileLink' column='family1' 2023-11-28 03:55:20,909 DEBUG [Listener at localhost/42853] zookeeper.ReadOnlyZKClient(139): Connect 0x388cfe85 to 127.0.0.1:55641 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-11-28 03:55:20,913 DEBUG [Listener at localhost/42853] ipc.AbstractRpcClient(189): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@78711ec9, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-11-28 03:55:20,913 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20231128 for table='testCompactionWithHFileLink' and column='family1' 2023-11-28 03:55:20,913 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c4ca4238a0b923820dcc509a6f75849b20231128 for table='testCompactionWithHFileLink' and column='family1' 2023-11-28 03:55:20,913 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c81e728d9d4c2f636f067f89cc14862c20231128 for table='testCompactionWithHFileLink' and column='family1' 2023-11-28 03:55:21,411 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023112829f0256d4d6f432d9570004e36d6c781 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023112829f0256d4d6f432d9570004e36d6c781 2023-11-28 03:55:21,412 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20231128 table='testCompactionWithHFileLink' column='family1' 2023-11-28 03:55:21,419 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202311280ca5b91d3f78438c85057d558cfb1fcd to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202311280ca5b91d3f78438c85057d558cfb1fcd 2023-11-28 03:55:21,420 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20231128 table='testCompactionWithHFileLink' column='family1' 2023-11-28 03:55:21,430 DEBUG [MobFileCompactionChore-pool-2] ipc.RpcConnection(122): Using SIMPLE authentication for service=ClientService, sasl=false 2023-11-28 03:55:21,434 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:55320, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-11-28 03:55:21,441 DEBUG [MobFileCompactionChore-pool-1] ipc.RpcConnection(122): Using SIMPLE authentication for service=MasterService, sasl=false 2023-11-28 03:55:21,445 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:39772, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-11-28 03:55:21,470 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20231128/family1/bd7aa4cb5d54428eae9cd8631eb70cf9 first=Optional[1\x00\x00\x00\x00] last=Optional[1\x00\x00\x00\x07] 2023-11-28 03:55:21,471 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testCompactionWithHFileLink,1,1701143695226.b3c2fdb68f55a3cbe8d29ee7b6ddd5e5., hostname=jenkins-hbase4.apache.org,41203,1701143600653, seqNum=32 for row 1 with hfile group [{family1,hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20231128/family1/bd7aa4cb5d54428eae9cd8631eb70cf9}] 2023-11-28 03:55:21,471 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20231128/family1/a739716e87bb4c278dfd9276590a07fb first=Optional[2\x00\x00\x00\x00] last=Optional[2\x00\x00\x00\x07] 2023-11-28 03:55:21,477 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/.tmp/family1/e46af3732459412182150c945b9633c8 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family1/e46af3732459412182150c945b9633c8 2023-11-28 03:55:21,484 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testCompactionWithHFileLink,2,1701143695226.2aad05dc9c73dcbea66259e5c0a6958f., hostname=jenkins-hbase4.apache.org,41203,1701143600653, seqNum=32 for row 2 with hfile group [{family1,hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20231128/family1/a739716e87bb4c278dfd9276590a07fb}] 2023-11-28 03:55:21,490 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41203] regionserver.HStore(763): Validating hfile at hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20231128/family1/bd7aa4cb5d54428eae9cd8631eb70cf9 for inclusion in b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family1 2023-11-28 03:55:21,490 INFO [RS:0;jenkins-hbase4:41203-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in 3d7f49de72f2e9e741f04714c6a7648c/family1 of 3d7f49de72f2e9e741f04714c6a7648c into e46af3732459412182150c945b9633c8(size=7.6 K), total size for store is 7.6 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-11-28 03:55:21,490 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 3d7f49de72f2e9e741f04714c6a7648c: 2023-11-28 03:55:21,490 INFO [RS:0;jenkins-hbase4:41203-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testCompactionWithHFileLink,,1701143695226.3d7f49de72f2e9e741f04714c6a7648c., storeName=3d7f49de72f2e9e741f04714c6a7648c/family1, priority=12, startTime=1701143720137; duration=1sec 2023-11-28 03:55:21,490 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=3), splitQueue=0 2023-11-28 03:55:21,493 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-11-28 03:55:21,494 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HStore(763): Validating hfile at hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20231128/family1/a739716e87bb4c278dfd9276590a07fb for inclusion in 2aad05dc9c73dcbea66259e5c0a6958f/family1 2023-11-28 03:55:21,498 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41203] regionserver.HStore(775): HFile bounds: first=1\x00\x00\x00\x00 last=1\x00\x00\x00\x07 2023-11-28 03:55:21,498 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41203] regionserver.HStore(777): Region bounds: first=1 last=2 2023-11-28 03:55:21,499 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HStore(775): HFile bounds: first=2\x00\x00\x00\x00 last=2\x00\x00\x00\x07 2023-11-28 03:55:21,499 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41203] regionserver.HRegion(2446): Flush status journal for b3c2fdb68f55a3cbe8d29ee7b6ddd5e5: 2023-11-28 03:55:21,499 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HStore(777): Region bounds: first=2 last= 2023-11-28 03:55:21,500 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HRegion(2446): Flush status journal for 2aad05dc9c73dcbea66259e5c0a6958f: 2023-11-28 03:55:21,500 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 4 files of size 20920 starting at candidate #0 after considering 3 permutations with 3 in ratio 2023-11-28 03:55:21,500 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] regionserver.HStore(1912): b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family2 is initiating minor compaction (all files) 2023-11-28 03:55:21,500 INFO [RS:0;jenkins-hbase4:41203-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family2 in testCompactionWithHFileLink,1,1701143695226.b3c2fdb68f55a3cbe8d29ee7b6ddd5e5. 2023-11-28 03:55:21,501 INFO [RS:0;jenkins-hbase4:41203-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family2/testCompactionWithHFileLink=b3c2fdb68f55a3cbe8d29ee7b6ddd5e5-ee2d50b684b2479e8a02236cf5083faa, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family2/testCompactionWithHFileLink=b3c2fdb68f55a3cbe8d29ee7b6ddd5e5-75ff1f4a28e9466e86a2f50fe4b3ba7c, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family2/testCompactionWithHFileLink=b3c2fdb68f55a3cbe8d29ee7b6ddd5e5-b06633c919b549ed9628414e3a0d7c29, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family2/testCompactionWithHFileLink=b3c2fdb68f55a3cbe8d29ee7b6ddd5e5-fe11679925e74afa9cd10ab81215b48a] into tmpdir=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/.tmp, totalSize=20.4 K 2023-11-28 03:55:21,501 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41203] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20231128/family1/bd7aa4cb5d54428eae9cd8631eb70cf9 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testCompactionWithHFileLink__q4qc5qb3itkdlhom7atbt35esjq5ts0f869q9b6aclkh8l2tphffmidn4a8lkidr/family1/bd7aa4cb5d54428eae9cd8631eb70cf9 2023-11-28 03:55:21,501 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20231128/family1/a739716e87bb4c278dfd9276590a07fb to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testCompactionWithHFileLink__vem95d4hb2utvoriea6m51vduqosr4o91pk4a1otmhekilu618gk0l43tkmm3ics/family1/a739716e87bb4c278dfd9276590a07fb 2023-11-28 03:55:21,501 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=b3c2fdb68f55a3cbe8d29ee7b6ddd5e5-ee2d50b684b2479e8a02236cf5083faa, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1701143697344 2023-11-28 03:55:21,502 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=b3c2fdb68f55a3cbe8d29ee7b6ddd5e5-75ff1f4a28e9466e86a2f50fe4b3ba7c, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1701143698739 2023-11-28 03:55:21,504 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=b3c2fdb68f55a3cbe8d29ee7b6ddd5e5-b06633c919b549ed9628414e3a0d7c29, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1701143700547 2023-11-28 03:55:21,504 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41203] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testCompactionWithHFileLink__q4qc5qb3itkdlhom7atbt35esjq5ts0f869q9b6aclkh8l2tphffmidn4a8lkidr/family1/bd7aa4cb5d54428eae9cd8631eb70cf9 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family1/f34c57f507ce46a8bf8c46a25c5472ca_SeqId_34_ 2023-11-28 03:55:21,504 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testCompactionWithHFileLink__vem95d4hb2utvoriea6m51vduqosr4o91pk4a1otmhekilu618gk0l43tkmm3ics/family1/a739716e87bb4c278dfd9276590a07fb as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family1/3f27129df1484f0f9be5d64a51ca9f4c_SeqId_34_ 2023-11-28 03:55:21,504 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41203] regionserver.HStore(849): Loaded HFile hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testCompactionWithHFileLink__q4qc5qb3itkdlhom7atbt35esjq5ts0f869q9b6aclkh8l2tphffmidn4a8lkidr/family1/bd7aa4cb5d54428eae9cd8631eb70cf9 into b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family1 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family1/f34c57f507ce46a8bf8c46a25c5472ca_SeqId_34_ - updating store file list. 2023-11-28 03:55:21,505 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family1/testCompactionWithHFileLink=3d7f49de72f2e9e741f04714c6a7648c-9cba99270b974d6b9181045f6f7ee6bc, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family1/testCompactionWithHFileLink=3d7f49de72f2e9e741f04714c6a7648c-149c75fc3d8a444bbd7d2180c2614583, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family1/testCompactionWithHFileLink=3d7f49de72f2e9e741f04714c6a7648c-b313b9e7ed474d04bf592f578c1928e8, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family1/testCompactionWithHFileLink=3d7f49de72f2e9e741f04714c6a7648c-0f49b13a3dc94f1bb9ed692753ce2e94] to archive 2023-11-28 03:55:21,507 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HStore(849): Loaded HFile hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testCompactionWithHFileLink__vem95d4hb2utvoriea6m51vduqosr4o91pk4a1otmhekilu618gk0l43tkmm3ics/family1/a739716e87bb4c278dfd9276590a07fb into 2aad05dc9c73dcbea66259e5c0a6958f/family1 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family1/3f27129df1484f0f9be5d64a51ca9f4c_SeqId_34_ - updating store file list. 2023-11-28 03:55:21,508 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-28 03:55:21,508 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=b3c2fdb68f55a3cbe8d29ee7b6ddd5e5-fe11679925e74afa9cd10ab81215b48a, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1701143702350 2023-11-28 03:55:21,511 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41203] regionserver.HStore(882): Loaded HFile hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family1/f34c57f507ce46a8bf8c46a25c5472ca_SeqId_34_ into b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family1 2023-11-28 03:55:21,511 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41203] regionserver.HStore(855): Successfully loaded hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testCompactionWithHFileLink__q4qc5qb3itkdlhom7atbt35esjq5ts0f869q9b6aclkh8l2tphffmidn4a8lkidr/family1/bd7aa4cb5d54428eae9cd8631eb70cf9 into b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family1 (new location: hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family1/f34c57f507ce46a8bf8c46a25c5472ca_SeqId_34_) 2023-11-28 03:55:21,511 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41203] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testCompactionWithHFileLink__q4qc5qb3itkdlhom7atbt35esjq5ts0f869q9b6aclkh8l2tphffmidn4a8lkidr/family1/bd7aa4cb5d54428eae9cd8631eb70cf9 2023-11-28 03:55:21,512 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41203] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-11-28 03:55:21,512 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family1/testCompactionWithHFileLink=3d7f49de72f2e9e741f04714c6a7648c-9cba99270b974d6b9181045f6f7ee6bc to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family1/testCompactionWithHFileLink=3d7f49de72f2e9e741f04714c6a7648c-9cba99270b974d6b9181045f6f7ee6bc 2023-11-28 03:55:21,514 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HStore(882): Loaded HFile hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family1/3f27129df1484f0f9be5d64a51ca9f4c_SeqId_34_ into 2aad05dc9c73dcbea66259e5c0a6958f/family1 2023-11-28 03:55:21,514 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HStore(855): Successfully loaded hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testCompactionWithHFileLink__vem95d4hb2utvoriea6m51vduqosr4o91pk4a1otmhekilu618gk0l43tkmm3ics/family1/a739716e87bb4c278dfd9276590a07fb into 2aad05dc9c73dcbea66259e5c0a6958f/family1 (new location: hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family1/3f27129df1484f0f9be5d64a51ca9f4c_SeqId_34_) 2023-11-28 03:55:21,514 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family1/testCompactionWithHFileLink=3d7f49de72f2e9e741f04714c6a7648c-149c75fc3d8a444bbd7d2180c2614583 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family1/testCompactionWithHFileLink=3d7f49de72f2e9e741f04714c6a7648c-149c75fc3d8a444bbd7d2180c2614583 2023-11-28 03:55:21,515 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testCompactionWithHFileLink__vem95d4hb2utvoriea6m51vduqosr4o91pk4a1otmhekilu618gk0l43tkmm3ics/family1/a739716e87bb4c278dfd9276590a07fb 2023-11-28 03:55:21,516 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family1/testCompactionWithHFileLink=3d7f49de72f2e9e741f04714c6a7648c-b313b9e7ed474d04bf592f578c1928e8 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family1/testCompactionWithHFileLink=3d7f49de72f2e9e741f04714c6a7648c-b313b9e7ed474d04bf592f578c1928e8 2023-11-28 03:55:21,517 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20231128 table='testCompactionWithHFileLink' column='family1' 2023-11-28 03:55:21,518 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41203] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 5 files of size 29606 starting at candidate #0 after considering 6 permutations with 6 in ratio 2023-11-28 03:55:21,518 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41203] regionserver.HStore(1912): b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family1 is initiating minor compaction (all files) 2023-11-28 03:55:21,518 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41203] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@367e390b; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=3), splitQueue=0 2023-11-28 03:55:21,519 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(360): Archiving compacted files. 2023-11-28 03:55:21,519 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family1/testCompactionWithHFileLink=3d7f49de72f2e9e741f04714c6a7648c-0f49b13a3dc94f1bb9ed692753ce2e94 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family1/testCompactionWithHFileLink=3d7f49de72f2e9e741f04714c6a7648c-0f49b13a3dc94f1bb9ed692753ce2e94 2023-11-28 03:55:21,521 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c20231128548ceb87df594da68e7f4d2121a8bbf7 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c20231128548ceb87df594da68e7f4d2121a8bbf7 2023-11-28 03:55:21,522 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20231128 table='testCompactionWithHFileLink' column='family1' 2023-11-28 03:55:21,523 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c20231128dec752e8fb644a0a80d6b4fd043402c1 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c20231128dec752e8fb644a0a80d6b4fd043402c1 2023-11-28 03:55:21,523 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(360): Archiving compacted files. 2023-11-28 03:55:21,525 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c20231128eb89ca1113e144ddaad3b48ac062b47f to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c20231128eb89ca1113e144ddaad3b48ac062b47f 2023-11-28 03:55:21,525 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b2023112838c9c1bb8cde4e7f815926db3a63a195 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b2023112838c9c1bb8cde4e7f815926db3a63a195 2023-11-28 03:55:21,526 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c20231128fb09572884984543b4a9faae21b5679c to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c20231128fb09572884984543b4a9faae21b5679c 2023-11-28 03:55:21,526 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-11-28 03:55:21,526 DEBUG [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20231128' 2023-11-28 03:55:21,527 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b2023112878e8d44c7cc44d39bafe2620db40b1d7 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b2023112878e8d44c7cc44d39bafe2620db40b1d7 2023-11-28 03:55:21,527 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c81e728d9d4c2f636f067f89cc14862c20231128 for table='testCompactionWithHFileLink' and column='family1' 2023-11-28 03:55:21,528 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b202311287fb879a012e040c992a5c4c12e52bff2 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b202311287fb879a012e040c992a5c4c12e52bff2 2023-11-28 03:55:21,530 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b20231128bd3184bcc4744bf591f18e1d0b9b0eb7 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b20231128bd3184bcc4744bf591f18e1d0b9b0eb7 2023-11-28 03:55:21,530 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-11-28 03:55:21,530 DEBUG [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20231128' 2023-11-28 03:55:21,530 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c4ca4238a0b923820dcc509a6f75849b20231128 for table='testCompactionWithHFileLink' and column='family1' 2023-11-28 03:55:21,533 INFO [RS:0;jenkins-hbase4:41203-shortCompactions-0] throttle.PressureAwareThroughputController(145): b3c2fdb68f55a3cbe8d29ee7b6ddd5e5#family2#compaction#116 average throughput is 0.39 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-11-28 03:55:21,562 DEBUG [RS:0;jenkins-hbase4:41203-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/.tmp/family1/8cad1c176c244ced9c512f1ef098f3e4 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family1/8cad1c176c244ced9c512f1ef098f3e4 2023-11-28 03:55:21,566 INFO [RS:0;jenkins-hbase4:41203-longCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in 2aad05dc9c73dcbea66259e5c0a6958f/family1 of 2aad05dc9c73dcbea66259e5c0a6958f into 8cad1c176c244ced9c512f1ef098f3e4(size=7.6 K), total size for store is 14.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-11-28 03:55:21,566 DEBUG [RS:0;jenkins-hbase4:41203-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 2aad05dc9c73dcbea66259e5c0a6958f: 2023-11-28 03:55:21,566 INFO [RS:0;jenkins-hbase4:41203-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testCompactionWithHFileLink,2,1701143695226.2aad05dc9c73dcbea66259e5c0a6958f., storeName=2aad05dc9c73dcbea66259e5c0a6958f/family1, priority=12, startTime=1701143720306; duration=0sec 2023-11-28 03:55:21,566 DEBUG [RS:0;jenkins-hbase4:41203-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=3), splitQueue=0 2023-11-28 03:55:21,566 INFO [RS:0;jenkins-hbase4:41203-longCompactions-0] regionserver.HRegion(2259): Starting compaction of b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family1 in testCompactionWithHFileLink,1,1701143695226.b3c2fdb68f55a3cbe8d29ee7b6ddd5e5. 2023-11-28 03:55:21,566 INFO [RS:0;jenkins-hbase4:41203-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family1/testCompactionWithHFileLink=b3c2fdb68f55a3cbe8d29ee7b6ddd5e5-88a76a3cbca146b6bb71214ecc4a9c87, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family1/testCompactionWithHFileLink=b3c2fdb68f55a3cbe8d29ee7b6ddd5e5-299d98e37656434ea5b2482137c58710, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family1/testCompactionWithHFileLink=b3c2fdb68f55a3cbe8d29ee7b6ddd5e5-dc7b41ef849648dda6e80bfb0cdeea0f, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family1/testCompactionWithHFileLink=b3c2fdb68f55a3cbe8d29ee7b6ddd5e5-34bdbacde3fd4331ae0cc21041140fd9, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family1/f34c57f507ce46a8bf8c46a25c5472ca_SeqId_34_] into tmpdir=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/.tmp, totalSize=28.9 K 2023-11-28 03:55:21,567 DEBUG [RS:0;jenkins-hbase4:41203-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=b3c2fdb68f55a3cbe8d29ee7b6ddd5e5-88a76a3cbca146b6bb71214ecc4a9c87, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1701143697344 2023-11-28 03:55:21,567 DEBUG [RS:0;jenkins-hbase4:41203-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=b3c2fdb68f55a3cbe8d29ee7b6ddd5e5-299d98e37656434ea5b2482137c58710, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1701143698739 2023-11-28 03:55:21,568 DEBUG [RS:0;jenkins-hbase4:41203-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=b3c2fdb68f55a3cbe8d29ee7b6ddd5e5-dc7b41ef849648dda6e80bfb0cdeea0f, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1701143700547 2023-11-28 03:55:21,568 DEBUG [RS:0;jenkins-hbase4:41203-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=b3c2fdb68f55a3cbe8d29ee7b6ddd5e5-34bdbacde3fd4331ae0cc21041140fd9, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1701143702350 2023-11-28 03:55:21,569 DEBUG [RS:0;jenkins-hbase4:41203-longCompactions-0] compactions.Compactor(207): Compacting f34c57f507ce46a8bf8c46a25c5472ca_SeqId_34_, keycount=16, bloomtype=ROW, size=7.3 K, encoding=NONE, compression=NONE, seqNum=35, earliestPutTs=1701143697344 2023-11-28 03:55:21,588 INFO [RS:0;jenkins-hbase4:41203-longCompactions-0] throttle.PressureAwareThroughputController(145): b3c2fdb68f55a3cbe8d29ee7b6ddd5e5#family1#compaction#117 average throughput is 0.33 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-11-28 03:55:21,605 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family1/testCompactionWithHFileLink=2aad05dc9c73dcbea66259e5c0a6958f-5bd4594355ca45ff98e17e8e298757d3, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family1/testCompactionWithHFileLink=2aad05dc9c73dcbea66259e5c0a6958f-ef8ca9f746b4468d8da6b892ab104927, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family1/testCompactionWithHFileLink=2aad05dc9c73dcbea66259e5c0a6958f-55b0944f49f248fd87ca616d0ed03e14, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family1/testCompactionWithHFileLink=2aad05dc9c73dcbea66259e5c0a6958f-ab034fef3fd24871aeeb0dc81dfe66c2] to archive 2023-11-28 03:55:21,606 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-28 03:55:21,607 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family1/testCompactionWithHFileLink=2aad05dc9c73dcbea66259e5c0a6958f-5bd4594355ca45ff98e17e8e298757d3 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family1/testCompactionWithHFileLink=2aad05dc9c73dcbea66259e5c0a6958f-5bd4594355ca45ff98e17e8e298757d3 2023-11-28 03:55:21,609 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family1/testCompactionWithHFileLink=2aad05dc9c73dcbea66259e5c0a6958f-ef8ca9f746b4468d8da6b892ab104927 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family1/testCompactionWithHFileLink=2aad05dc9c73dcbea66259e5c0a6958f-ef8ca9f746b4468d8da6b892ab104927 2023-11-28 03:55:21,610 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family1/testCompactionWithHFileLink=2aad05dc9c73dcbea66259e5c0a6958f-55b0944f49f248fd87ca616d0ed03e14 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family1/testCompactionWithHFileLink=2aad05dc9c73dcbea66259e5c0a6958f-55b0944f49f248fd87ca616d0ed03e14 2023-11-28 03:55:21,611 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family1/testCompactionWithHFileLink=2aad05dc9c73dcbea66259e5c0a6958f-ab034fef3fd24871aeeb0dc81dfe66c2 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family1/testCompactionWithHFileLink=2aad05dc9c73dcbea66259e5c0a6958f-ab034fef3fd24871aeeb0dc81dfe66c2 2023-11-28 03:55:21,812 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20231128d6cdd61c280545e39c6de52830954d2f to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20231128d6cdd61c280545e39c6de52830954d2f 2023-11-28 03:55:21,813 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20231128 table='testCompactionWithHFileLink' column='family1' 2023-11-28 03:55:21,847 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20231128/family1/cfcbfb4299e3431aaac47da2fb48e22f first=Optional[0\x00\x00\x00\x00] last=Optional[0\x00\x00\x00\x07] 2023-11-28 03:55:21,848 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testCompactionWithHFileLink,,1701143695226.3d7f49de72f2e9e741f04714c6a7648c., hostname=jenkins-hbase4.apache.org,41203,1701143600653, seqNum=32 for row with hfile group [{family1,hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20231128/family1/cfcbfb4299e3431aaac47da2fb48e22f}] 2023-11-28 03:55:21,850 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41203] regionserver.HStore(763): Validating hfile at hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20231128/family1/cfcbfb4299e3431aaac47da2fb48e22f for inclusion in 3d7f49de72f2e9e741f04714c6a7648c/family1 2023-11-28 03:55:21,854 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41203] regionserver.HStore(775): HFile bounds: first=0\x00\x00\x00\x00 last=0\x00\x00\x00\x07 2023-11-28 03:55:21,854 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41203] regionserver.HStore(777): Region bounds: first= last=1 2023-11-28 03:55:21,854 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41203] regionserver.HRegion(2446): Flush status journal for 3d7f49de72f2e9e741f04714c6a7648c: 2023-11-28 03:55:21,855 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41203] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20231128/family1/cfcbfb4299e3431aaac47da2fb48e22f to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testCompactionWithHFileLink__hdqsqf8jj7r4036qajt8cru8667ks2s34tggt12m4tp9v9asefjnrhhv8l3hojml/family1/cfcbfb4299e3431aaac47da2fb48e22f 2023-11-28 03:55:21,857 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41203] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testCompactionWithHFileLink__hdqsqf8jj7r4036qajt8cru8667ks2s34tggt12m4tp9v9asefjnrhhv8l3hojml/family1/cfcbfb4299e3431aaac47da2fb48e22f as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family1/65f2ea90e6e14b1492f12b47d39e00ca_SeqId_36_ 2023-11-28 03:55:21,857 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41203] regionserver.HStore(849): Loaded HFile hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testCompactionWithHFileLink__hdqsqf8jj7r4036qajt8cru8667ks2s34tggt12m4tp9v9asefjnrhhv8l3hojml/family1/cfcbfb4299e3431aaac47da2fb48e22f into 3d7f49de72f2e9e741f04714c6a7648c/family1 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family1/65f2ea90e6e14b1492f12b47d39e00ca_SeqId_36_ - updating store file list. 2023-11-28 03:55:21,861 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41203] regionserver.HStore(882): Loaded HFile hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family1/65f2ea90e6e14b1492f12b47d39e00ca_SeqId_36_ into 3d7f49de72f2e9e741f04714c6a7648c/family1 2023-11-28 03:55:21,861 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41203] regionserver.HStore(855): Successfully loaded hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testCompactionWithHFileLink__hdqsqf8jj7r4036qajt8cru8667ks2s34tggt12m4tp9v9asefjnrhhv8l3hojml/family1/cfcbfb4299e3431aaac47da2fb48e22f into 3d7f49de72f2e9e741f04714c6a7648c/family1 (new location: hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/family1/65f2ea90e6e14b1492f12b47d39e00ca_SeqId_36_) 2023-11-28 03:55:21,861 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41203] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testCompactionWithHFileLink__hdqsqf8jj7r4036qajt8cru8667ks2s34tggt12m4tp9v9asefjnrhhv8l3hojml/family1/cfcbfb4299e3431aaac47da2fb48e22f 2023-11-28 03:55:21,863 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20231128 table='testCompactionWithHFileLink' column='family1' 2023-11-28 03:55:21,864 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-28 03:55:21,865 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e202311280eef9bc4c106435986d8700c2b7868aa to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e202311280eef9bc4c106435986d8700c2b7868aa 2023-11-28 03:55:21,866 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e202311285c06394279144f2aa2ca5a78e6ddb299 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e202311285c06394279144f2aa2ca5a78e6ddb299 2023-11-28 03:55:21,867 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e2023112864ca880632ff4b81b856e56f97a51ba5 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e2023112864ca880632ff4b81b856e56f97a51ba5 2023-11-28 03:55:21,868 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e20231128ae21d695d78b47819031a63d23fd1adf to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e20231128ae21d695d78b47819031a63d23fd1adf 2023-11-28 03:55:21,868 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-11-28 03:55:21,869 DEBUG [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20231128' 2023-11-28 03:55:21,869 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20231128 for table='testCompactionWithHFileLink' and column='family1' 2023-11-28 03:55:21,869 INFO [Listener at localhost/42853] compactions.PartitionedMobCompactor(354): After compaction, there are 3 mob files. table='testCompactionWithHFileLink' column='family1' 2023-11-28 03:55:21,873 DEBUG [Listener at localhost/42853] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-11-28 03:55:21,875 DEBUG [regionserver/jenkins-hbase4:0.Chore.1] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.15384615384615385, tune throughput to 57.69 MB/second 2023-11-28 03:55:21,875 DEBUG [Listener at localhost/42853] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-11-28 03:55:21,880 DEBUG [Listener at localhost/42853] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-11-28 03:55:21,881 DEBUG [Listener at localhost/42853] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-11-28 03:55:21,889 DEBUG [Listener at localhost/42853] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-11-28 03:55:21,890 DEBUG [Listener at localhost/42853] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-11-28 03:55:21,907 INFO [Listener at localhost/42853] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testCompactionWithHFileLink Thread=286 (was 240) - Thread LEAK? -, OpenFileDescriptor=771 (was 627) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=196 (was 164) - SystemLoadAverage LEAK? -, ProcessCount=168 (was 168), AvailableMemoryMB=7035 (was 6849) - AvailableMemoryMB LEAK? - 2023-11-28 03:55:21,919 INFO [Listener at localhost/42853] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMinorCompaction Thread=286, OpenFileDescriptor=771, MaxFileDescriptor=60000, SystemLoadAverage=196, ProcessCount=168, AvailableMemoryMB=7034 2023-11-28 03:55:21,925 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] master.HMaster$15(3014): Client=jenkins//172.31.14.131 creating {NAME => 'ns'} 2023-11-28 03:55:21,926 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] procedure2.ProcedureExecutor(1028): Stored pid=104, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=ns 2023-11-28 03:55:21,930 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] master.MasterRpcServices(1230): Checking to see if procedure is done pid=104 2023-11-28 03:55:21,933 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-11-28 03:55:21,936 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=104, state=SUCCESS; CreateNamespaceProcedure, namespace=ns in 10 msec 2023-11-28 03:55:22,031 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] master.MasterRpcServices(1230): Checking to see if procedure is done pid=104 2023-11-28 03:55:22,033 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'ns:testMinorCompaction', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} 2023-11-28 03:55:22,035 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] procedure2.ProcedureExecutor(1028): Stored pid=105, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=ns:testMinorCompaction 2023-11-28 03:55:22,037 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_PRE_OPERATION 2023-11-28 03:55:22,037 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] master.MasterRpcServices(700): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "ns" qualifier: "testMinorCompaction" procId is: 105 2023-11-28 03:55:22,037 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-11-28 03:55:22,038 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] master.MasterRpcServices(1230): Checking to see if procedure is done pid=105 2023-11-28 03:55:22,039 DEBUG [HFileArchiver-16] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/.tmp/data/ns/testMinorCompaction/8f7c6ece6778b5f8b473ef7daf18a3c4 2023-11-28 03:55:22,040 DEBUG [HFileArchiver-17] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/.tmp/data/ns/testMinorCompaction/227f7071ca3e7b6612f5a149eeb75e98 2023-11-28 03:55:22,040 DEBUG [HFileArchiver-18] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/.tmp/data/ns/testMinorCompaction/d38dce7d1890ef61d80a87d7305594c5 2023-11-28 03:55:22,040 DEBUG [HFileArchiver-16] backup.HFileArchiver(153): Directory hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/.tmp/data/ns/testMinorCompaction/8f7c6ece6778b5f8b473ef7daf18a3c4 empty. 2023-11-28 03:55:22,040 DEBUG [HFileArchiver-17] backup.HFileArchiver(153): Directory hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/.tmp/data/ns/testMinorCompaction/227f7071ca3e7b6612f5a149eeb75e98 empty. 2023-11-28 03:55:22,040 DEBUG [HFileArchiver-18] backup.HFileArchiver(153): Directory hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/.tmp/data/ns/testMinorCompaction/d38dce7d1890ef61d80a87d7305594c5 empty. 2023-11-28 03:55:22,040 DEBUG [HFileArchiver-16] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/.tmp/data/ns/testMinorCompaction/8f7c6ece6778b5f8b473ef7daf18a3c4 2023-11-28 03:55:22,040 DEBUG [HFileArchiver-17] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/.tmp/data/ns/testMinorCompaction/227f7071ca3e7b6612f5a149eeb75e98 2023-11-28 03:55:22,041 DEBUG [HFileArchiver-18] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/.tmp/data/ns/testMinorCompaction/d38dce7d1890ef61d80a87d7305594c5 2023-11-28 03:55:22,041 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(328): Archived ns:testMinorCompaction regions 2023-11-28 03:55:22,080 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/.tmp/data/ns/testMinorCompaction/.tabledesc/.tableinfo.0000000001 2023-11-28 03:55:22,086 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(7675): creating {ENCODED => 227f7071ca3e7b6612f5a149eeb75e98, NAME => 'ns:testMinorCompaction,1,1701143722033.227f7071ca3e7b6612f5a149eeb75e98.', STARTKEY => '1', ENDKEY => '2'}, tableDescriptor='ns:testMinorCompaction', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/.tmp 2023-11-28 03:55:22,086 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(7675): creating {ENCODED => 8f7c6ece6778b5f8b473ef7daf18a3c4, NAME => 'ns:testMinorCompaction,,1701143722033.8f7c6ece6778b5f8b473ef7daf18a3c4.', STARTKEY => '', ENDKEY => '1'}, tableDescriptor='ns:testMinorCompaction', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/.tmp 2023-11-28 03:55:22,088 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(7675): creating {ENCODED => d38dce7d1890ef61d80a87d7305594c5, NAME => 'ns:testMinorCompaction,2,1701143722033.d38dce7d1890ef61d80a87d7305594c5.', STARTKEY => '2', ENDKEY => ''}, tableDescriptor='ns:testMinorCompaction', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/.tmp 2023-11-28 03:55:22,139 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] master.MasterRpcServices(1230): Checking to see if procedure is done pid=105 2023-11-28 03:55:22,340 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] master.MasterRpcServices(1230): Checking to see if procedure is done pid=105 2023-11-28 03:55:22,355 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-11-28 03:55:22,367 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/.tmp/family2/550d7e07bda249c0835d0089913f0332 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family2/550d7e07bda249c0835d0089913f0332 2023-11-28 03:55:22,372 INFO [RS:0;jenkins-hbase4:41203-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family2 of b3c2fdb68f55a3cbe8d29ee7b6ddd5e5 into 550d7e07bda249c0835d0089913f0332(size=6.4 K), total size for store is 6.4 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-11-28 03:55:22,373 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for b3c2fdb68f55a3cbe8d29ee7b6ddd5e5: 2023-11-28 03:55:22,373 INFO [RS:0;jenkins-hbase4:41203-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testCompactionWithHFileLink,1,1701143695226.b3c2fdb68f55a3cbe8d29ee7b6ddd5e5., storeName=b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family2, priority=12, startTime=1701143720378; duration=0sec 2023-11-28 03:55:22,373 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=2), splitQueue=0 2023-11-28 03:55:22,373 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 5 compacting, 0 eligible, 16 blocking 2023-11-28 03:55:22,373 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 0 files of size 0 starting at candidate #-1 after considering 0 permutations with 0 in ratio 2023-11-28 03:55:22,373 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] compactions.SortedCompactionPolicy(232): Not compacting files because we only have 0 files ready for compaction. Need 3 to initiate. 2023-11-28 03:55:22,373 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] regionserver.CompactSplit(415): Not compacting testCompactionWithHFileLink,1,1701143695226.b3c2fdb68f55a3cbe8d29ee7b6ddd5e5. because compaction request was cancelled 2023-11-28 03:55:22,373 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-11-28 03:55:22,376 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 4 files of size 20920 starting at candidate #0 after considering 3 permutations with 3 in ratio 2023-11-28 03:55:22,376 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] regionserver.HStore(1912): 2aad05dc9c73dcbea66259e5c0a6958f/family2 is initiating minor compaction (all files) 2023-11-28 03:55:22,376 INFO [RS:0;jenkins-hbase4:41203-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 2aad05dc9c73dcbea66259e5c0a6958f/family2 in testCompactionWithHFileLink,2,1701143695226.2aad05dc9c73dcbea66259e5c0a6958f. 2023-11-28 03:55:22,376 INFO [RS:0;jenkins-hbase4:41203-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family2/testCompactionWithHFileLink=2aad05dc9c73dcbea66259e5c0a6958f-a79a53b7062d4e53b9199b02c23f6c24, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family2/testCompactionWithHFileLink=2aad05dc9c73dcbea66259e5c0a6958f-a5d2b1b5f59d4745beda10e8ec8e729f, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family2/testCompactionWithHFileLink=2aad05dc9c73dcbea66259e5c0a6958f-9d803becce0f4ac4ad8171434ff9b0a2, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family2/testCompactionWithHFileLink=2aad05dc9c73dcbea66259e5c0a6958f-cdc6a50b83e147878684e9d41c01c3e6] into tmpdir=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/.tmp, totalSize=20.4 K 2023-11-28 03:55:22,377 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=2aad05dc9c73dcbea66259e5c0a6958f-a79a53b7062d4e53b9199b02c23f6c24, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1701143697344 2023-11-28 03:55:22,378 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=2aad05dc9c73dcbea66259e5c0a6958f-a5d2b1b5f59d4745beda10e8ec8e729f, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1701143698739 2023-11-28 03:55:22,378 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=2aad05dc9c73dcbea66259e5c0a6958f-9d803becce0f4ac4ad8171434ff9b0a2, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1701143700547 2023-11-28 03:55:22,379 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=2aad05dc9c73dcbea66259e5c0a6958f-cdc6a50b83e147878684e9d41c01c3e6, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1701143702350 2023-11-28 03:55:22,401 INFO [RS:0;jenkins-hbase4:41203-shortCompactions-0] throttle.PressureAwareThroughputController(145): 2aad05dc9c73dcbea66259e5c0a6958f#family2#compaction#118 average throughput is 0.19 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 57.69 MB/second 2023-11-28 03:55:22,407 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family2/testCompactionWithHFileLink=b3c2fdb68f55a3cbe8d29ee7b6ddd5e5-ee2d50b684b2479e8a02236cf5083faa, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family2/testCompactionWithHFileLink=b3c2fdb68f55a3cbe8d29ee7b6ddd5e5-75ff1f4a28e9466e86a2f50fe4b3ba7c, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family2/testCompactionWithHFileLink=b3c2fdb68f55a3cbe8d29ee7b6ddd5e5-b06633c919b549ed9628414e3a0d7c29, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family2/testCompactionWithHFileLink=b3c2fdb68f55a3cbe8d29ee7b6ddd5e5-fe11679925e74afa9cd10ab81215b48a] to archive 2023-11-28 03:55:22,408 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-28 03:55:22,409 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family2/testCompactionWithHFileLink=b3c2fdb68f55a3cbe8d29ee7b6ddd5e5-ee2d50b684b2479e8a02236cf5083faa to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family2/testCompactionWithHFileLink=b3c2fdb68f55a3cbe8d29ee7b6ddd5e5-ee2d50b684b2479e8a02236cf5083faa 2023-11-28 03:55:22,411 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family2/testCompactionWithHFileLink=b3c2fdb68f55a3cbe8d29ee7b6ddd5e5-75ff1f4a28e9466e86a2f50fe4b3ba7c to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family2/testCompactionWithHFileLink=b3c2fdb68f55a3cbe8d29ee7b6ddd5e5-75ff1f4a28e9466e86a2f50fe4b3ba7c 2023-11-28 03:55:22,412 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family2/testCompactionWithHFileLink=b3c2fdb68f55a3cbe8d29ee7b6ddd5e5-b06633c919b549ed9628414e3a0d7c29 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family2/testCompactionWithHFileLink=b3c2fdb68f55a3cbe8d29ee7b6ddd5e5-b06633c919b549ed9628414e3a0d7c29 2023-11-28 03:55:22,414 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family2/testCompactionWithHFileLink=b3c2fdb68f55a3cbe8d29ee7b6ddd5e5-fe11679925e74afa9cd10ab81215b48a to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family2/testCompactionWithHFileLink=b3c2fdb68f55a3cbe8d29ee7b6ddd5e5-fe11679925e74afa9cd10ab81215b48a 2023-11-28 03:55:22,501 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(866): Instantiated ns:testMinorCompaction,1,1701143722033.227f7071ca3e7b6612f5a149eeb75e98.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-28 03:55:22,501 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1604): Closing 227f7071ca3e7b6612f5a149eeb75e98, disabling compactions & flushes 2023-11-28 03:55:22,501 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1626): Closing region ns:testMinorCompaction,1,1701143722033.227f7071ca3e7b6612f5a149eeb75e98. 2023-11-28 03:55:22,501 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1647): Waiting without time limit for close lock on ns:testMinorCompaction,1,1701143722033.227f7071ca3e7b6612f5a149eeb75e98. 2023-11-28 03:55:22,501 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1714): Acquired close lock on ns:testMinorCompaction,1,1701143722033.227f7071ca3e7b6612f5a149eeb75e98. after waiting 0 ms 2023-11-28 03:55:22,501 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1724): Updates disabled for region ns:testMinorCompaction,1,1701143722033.227f7071ca3e7b6612f5a149eeb75e98. 2023-11-28 03:55:22,501 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1838): Closed ns:testMinorCompaction,1,1701143722033.227f7071ca3e7b6612f5a149eeb75e98. 2023-11-28 03:55:22,501 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1558): Region close journal for 227f7071ca3e7b6612f5a149eeb75e98: 2023-11-28 03:55:22,514 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(866): Instantiated ns:testMinorCompaction,2,1701143722033.d38dce7d1890ef61d80a87d7305594c5.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-28 03:55:22,514 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1604): Closing d38dce7d1890ef61d80a87d7305594c5, disabling compactions & flushes 2023-11-28 03:55:22,514 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1626): Closing region ns:testMinorCompaction,2,1701143722033.d38dce7d1890ef61d80a87d7305594c5. 2023-11-28 03:55:22,514 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(866): Instantiated ns:testMinorCompaction,,1701143722033.8f7c6ece6778b5f8b473ef7daf18a3c4.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-28 03:55:22,514 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1647): Waiting without time limit for close lock on ns:testMinorCompaction,2,1701143722033.d38dce7d1890ef61d80a87d7305594c5. 2023-11-28 03:55:22,514 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1604): Closing 8f7c6ece6778b5f8b473ef7daf18a3c4, disabling compactions & flushes 2023-11-28 03:55:22,514 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1714): Acquired close lock on ns:testMinorCompaction,2,1701143722033.d38dce7d1890ef61d80a87d7305594c5. after waiting 0 ms 2023-11-28 03:55:22,514 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1724): Updates disabled for region ns:testMinorCompaction,2,1701143722033.d38dce7d1890ef61d80a87d7305594c5. 2023-11-28 03:55:22,514 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1626): Closing region ns:testMinorCompaction,,1701143722033.8f7c6ece6778b5f8b473ef7daf18a3c4. 2023-11-28 03:55:22,514 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1838): Closed ns:testMinorCompaction,2,1701143722033.d38dce7d1890ef61d80a87d7305594c5. 2023-11-28 03:55:22,514 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on ns:testMinorCompaction,,1701143722033.8f7c6ece6778b5f8b473ef7daf18a3c4. 2023-11-28 03:55:22,514 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1558): Region close journal for d38dce7d1890ef61d80a87d7305594c5: 2023-11-28 03:55:22,514 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1714): Acquired close lock on ns:testMinorCompaction,,1701143722033.8f7c6ece6778b5f8b473ef7daf18a3c4. after waiting 0 ms 2023-11-28 03:55:22,515 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1724): Updates disabled for region ns:testMinorCompaction,,1701143722033.8f7c6ece6778b5f8b473ef7daf18a3c4. 2023-11-28 03:55:22,515 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1838): Closed ns:testMinorCompaction,,1701143722033.8f7c6ece6778b5f8b473ef7daf18a3c4. 2023-11-28 03:55:22,515 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1558): Region close journal for 8f7c6ece6778b5f8b473ef7daf18a3c4: 2023-11-28 03:55:22,517 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_ADD_TO_META 2023-11-28 03:55:22,518 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"ns:testMinorCompaction,1,1701143722033.227f7071ca3e7b6612f5a149eeb75e98.","families":{"info":[{"qualifier":"regioninfo","vlen":50,"tag":[],"timestamp":"1701143722518"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1701143722518"}]},"ts":"1701143722518"} 2023-11-28 03:55:22,518 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"ns:testMinorCompaction,2,1701143722033.d38dce7d1890ef61d80a87d7305594c5.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1701143722518"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1701143722518"}]},"ts":"1701143722518"} 2023-11-28 03:55:22,518 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"ns:testMinorCompaction,,1701143722033.8f7c6ece6778b5f8b473ef7daf18a3c4.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1701143722518"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1701143722518"}]},"ts":"1701143722518"} 2023-11-28 03:55:22,520 INFO [PEWorker-2] hbase.MetaTableAccessor(1496): Added 3 regions to meta. 2023-11-28 03:55:22,520 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-11-28 03:55:22,521 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"ns:testMinorCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1701143722520"}]},"ts":"1701143722520"} 2023-11-28 03:55:22,522 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=ns:testMinorCompaction, state=ENABLING in hbase:meta 2023-11-28 03:55:22,528 INFO [PEWorker-2] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=8f7c6ece6778b5f8b473ef7daf18a3c4, ASSIGN}, {pid=107, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=227f7071ca3e7b6612f5a149eeb75e98, ASSIGN}, {pid=108, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=d38dce7d1890ef61d80a87d7305594c5, ASSIGN}] 2023-11-28 03:55:22,530 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=107, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=227f7071ca3e7b6612f5a149eeb75e98, ASSIGN 2023-11-28 03:55:22,530 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=8f7c6ece6778b5f8b473ef7daf18a3c4, ASSIGN 2023-11-28 03:55:22,530 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=108, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=d38dce7d1890ef61d80a87d7305594c5, ASSIGN 2023-11-28 03:55:22,531 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=107, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=ns:testMinorCompaction, region=227f7071ca3e7b6612f5a149eeb75e98, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,41203,1701143600653; forceNewPlan=false, retain=false 2023-11-28 03:55:22,531 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=ns:testMinorCompaction, region=8f7c6ece6778b5f8b473ef7daf18a3c4, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,41203,1701143600653; forceNewPlan=false, retain=false 2023-11-28 03:55:22,531 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=108, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=ns:testMinorCompaction, region=d38dce7d1890ef61d80a87d7305594c5, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,41203,1701143600653; forceNewPlan=false, retain=false 2023-11-28 03:55:22,641 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] master.MasterRpcServices(1230): Checking to see if procedure is done pid=105 2023-11-28 03:55:22,683 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=107 updating hbase:meta row=227f7071ca3e7b6612f5a149eeb75e98, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:22,683 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=106 updating hbase:meta row=8f7c6ece6778b5f8b473ef7daf18a3c4, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:22,683 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=108 updating hbase:meta row=d38dce7d1890ef61d80a87d7305594c5, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:22,683 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"ns:testMinorCompaction,1,1701143722033.227f7071ca3e7b6612f5a149eeb75e98.","families":{"info":[{"qualifier":"regioninfo","vlen":50,"tag":[],"timestamp":"1701143722683"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1701143722683"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1701143722683"}]},"ts":"1701143722683"} 2023-11-28 03:55:22,683 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"ns:testMinorCompaction,2,1701143722033.d38dce7d1890ef61d80a87d7305594c5.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1701143722683"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1701143722683"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1701143722683"}]},"ts":"1701143722683"} 2023-11-28 03:55:22,683 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"ns:testMinorCompaction,,1701143722033.8f7c6ece6778b5f8b473ef7daf18a3c4.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1701143722683"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1701143722683"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1701143722683"}]},"ts":"1701143722683"} 2023-11-28 03:55:22,685 INFO [PEWorker-5] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=109, ppid=107, state=RUNNABLE; OpenRegionProcedure 227f7071ca3e7b6612f5a149eeb75e98, server=jenkins-hbase4.apache.org,41203,1701143600653}] 2023-11-28 03:55:22,685 INFO [PEWorker-2] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=110, ppid=108, state=RUNNABLE; OpenRegionProcedure d38dce7d1890ef61d80a87d7305594c5, server=jenkins-hbase4.apache.org,41203,1701143600653}] 2023-11-28 03:55:22,685 INFO [PEWorker-3] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=111, ppid=106, state=RUNNABLE; OpenRegionProcedure 8f7c6ece6778b5f8b473ef7daf18a3c4, server=jenkins-hbase4.apache.org,41203,1701143600653}] 2023-11-28 03:55:22,818 DEBUG [RS:0;jenkins-hbase4:41203-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/.tmp/family1/d70e389e236c4877b4999f4241c99b51 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family1/d70e389e236c4877b4999f4241c99b51 2023-11-28 03:55:22,823 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/.tmp/family2/586d8deb45674995915fbe22cf9956f1 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family2/586d8deb45674995915fbe22cf9956f1 2023-11-28 03:55:22,825 INFO [RS:0;jenkins-hbase4:41203-longCompactions-0] regionserver.HStore(1652): Completed compaction of 5 (all) file(s) in b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family1 of b3c2fdb68f55a3cbe8d29ee7b6ddd5e5 into d70e389e236c4877b4999f4241c99b51(size=7.6 K), total size for store is 7.6 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-11-28 03:55:22,825 DEBUG [RS:0;jenkins-hbase4:41203-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for b3c2fdb68f55a3cbe8d29ee7b6ddd5e5: 2023-11-28 03:55:22,825 INFO [RS:0;jenkins-hbase4:41203-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testCompactionWithHFileLink,1,1701143695226.b3c2fdb68f55a3cbe8d29ee7b6ddd5e5., storeName=family1, fileCount=5, fileSize=28.9 K (5.4 K, 5.4 K, 5.4 K, 5.4 K, 7.3 K), priority=2, time=1701143721518; duration=1sec 2023-11-28 03:55:22,825 DEBUG [RS:0;jenkins-hbase4:41203-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-28 03:55:22,827 INFO [RS:0;jenkins-hbase4:41203-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in 2aad05dc9c73dcbea66259e5c0a6958f/family2 of 2aad05dc9c73dcbea66259e5c0a6958f into 586d8deb45674995915fbe22cf9956f1(size=6.4 K), total size for store is 6.4 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-11-28 03:55:22,827 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 2aad05dc9c73dcbea66259e5c0a6958f: 2023-11-28 03:55:22,827 INFO [RS:0;jenkins-hbase4:41203-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testCompactionWithHFileLink,2,1701143695226.2aad05dc9c73dcbea66259e5c0a6958f., storeName=2aad05dc9c73dcbea66259e5c0a6958f/family2, priority=12, startTime=1701143720306; duration=0sec 2023-11-28 03:55:22,827 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-28 03:55:22,836 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:22,838 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open ns:testMinorCompaction,1,1701143722033.227f7071ca3e7b6612f5a149eeb75e98. 2023-11-28 03:55:22,838 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 227f7071ca3e7b6612f5a149eeb75e98, NAME => 'ns:testMinorCompaction,1,1701143722033.227f7071ca3e7b6612f5a149eeb75e98.', STARTKEY => '1', ENDKEY => '2'} 2023-11-28 03:55:22,839 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMinorCompaction 227f7071ca3e7b6612f5a149eeb75e98 2023-11-28 03:55:22,839 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated ns:testMinorCompaction,1,1701143722033.227f7071ca3e7b6612f5a149eeb75e98.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-28 03:55:22,839 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 227f7071ca3e7b6612f5a149eeb75e98 2023-11-28 03:55:22,839 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 227f7071ca3e7b6612f5a149eeb75e98 2023-11-28 03:55:22,840 INFO [StoreOpener-227f7071ca3e7b6612f5a149eeb75e98-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 227f7071ca3e7b6612f5a149eeb75e98 2023-11-28 03:55:22,841 DEBUG [StoreOpener-227f7071ca3e7b6612f5a149eeb75e98-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/227f7071ca3e7b6612f5a149eeb75e98/family1 2023-11-28 03:55:22,841 DEBUG [StoreOpener-227f7071ca3e7b6612f5a149eeb75e98-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/227f7071ca3e7b6612f5a149eeb75e98/family1 2023-11-28 03:55:22,841 INFO [StoreOpener-227f7071ca3e7b6612f5a149eeb75e98-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 227f7071ca3e7b6612f5a149eeb75e98 columnFamilyName family1 2023-11-28 03:55:22,842 INFO [StoreOpener-227f7071ca3e7b6612f5a149eeb75e98-1] regionserver.HStore(310): Store=227f7071ca3e7b6612f5a149eeb75e98/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-28 03:55:22,843 INFO [StoreOpener-227f7071ca3e7b6612f5a149eeb75e98-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 227f7071ca3e7b6612f5a149eeb75e98 2023-11-28 03:55:22,844 DEBUG [StoreOpener-227f7071ca3e7b6612f5a149eeb75e98-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/227f7071ca3e7b6612f5a149eeb75e98/family2 2023-11-28 03:55:22,844 DEBUG [StoreOpener-227f7071ca3e7b6612f5a149eeb75e98-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/227f7071ca3e7b6612f5a149eeb75e98/family2 2023-11-28 03:55:22,844 INFO [StoreOpener-227f7071ca3e7b6612f5a149eeb75e98-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 227f7071ca3e7b6612f5a149eeb75e98 columnFamilyName family2 2023-11-28 03:55:22,847 INFO [StoreOpener-227f7071ca3e7b6612f5a149eeb75e98-1] regionserver.HStore(310): Store=227f7071ca3e7b6612f5a149eeb75e98/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-28 03:55:22,848 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/227f7071ca3e7b6612f5a149eeb75e98 2023-11-28 03:55:22,848 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/227f7071ca3e7b6612f5a149eeb75e98 2023-11-28 03:55:22,850 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table ns:testMinorCompaction descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-11-28 03:55:22,851 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 227f7071ca3e7b6612f5a149eeb75e98 2023-11-28 03:55:22,852 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/227f7071ca3e7b6612f5a149eeb75e98/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-11-28 03:55:22,853 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 227f7071ca3e7b6612f5a149eeb75e98; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=75127075, jitterRate=0.11948065459728241}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-11-28 03:55:22,853 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 227f7071ca3e7b6612f5a149eeb75e98: 2023-11-28 03:55:22,853 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2339): Post open deploy tasks for ns:testMinorCompaction,1,1701143722033.227f7071ca3e7b6612f5a149eeb75e98., pid=109, masterSystemTime=1701143722836 2023-11-28 03:55:22,854 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2366): Finished post open deploy task for ns:testMinorCompaction,1,1701143722033.227f7071ca3e7b6612f5a149eeb75e98. 2023-11-28 03:55:22,854 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened ns:testMinorCompaction,1,1701143722033.227f7071ca3e7b6612f5a149eeb75e98. 2023-11-28 03:55:22,855 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open ns:testMinorCompaction,2,1701143722033.d38dce7d1890ef61d80a87d7305594c5. 2023-11-28 03:55:22,855 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => d38dce7d1890ef61d80a87d7305594c5, NAME => 'ns:testMinorCompaction,2,1701143722033.d38dce7d1890ef61d80a87d7305594c5.', STARTKEY => '2', ENDKEY => ''} 2023-11-28 03:55:22,855 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=107 updating hbase:meta row=227f7071ca3e7b6612f5a149eeb75e98, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:22,855 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMinorCompaction d38dce7d1890ef61d80a87d7305594c5 2023-11-28 03:55:22,855 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"ns:testMinorCompaction,1,1701143722033.227f7071ca3e7b6612f5a149eeb75e98.","families":{"info":[{"qualifier":"regioninfo","vlen":50,"tag":[],"timestamp":"1701143722855"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1701143722855"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1701143722855"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1701143722855"}]},"ts":"1701143722855"} 2023-11-28 03:55:22,855 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated ns:testMinorCompaction,2,1701143722033.d38dce7d1890ef61d80a87d7305594c5.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-28 03:55:22,855 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for d38dce7d1890ef61d80a87d7305594c5 2023-11-28 03:55:22,855 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for d38dce7d1890ef61d80a87d7305594c5 2023-11-28 03:55:22,856 INFO [StoreOpener-d38dce7d1890ef61d80a87d7305594c5-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region d38dce7d1890ef61d80a87d7305594c5 2023-11-28 03:55:22,857 INFO [PEWorker-2] procedure2.ProcedureExecutor(1825): Finished subprocedure pid=109, resume processing ppid=107 2023-11-28 03:55:22,858 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=109, ppid=107, state=SUCCESS; OpenRegionProcedure 227f7071ca3e7b6612f5a149eeb75e98, server=jenkins-hbase4.apache.org,41203,1701143600653 in 171 msec 2023-11-28 03:55:22,858 DEBUG [StoreOpener-d38dce7d1890ef61d80a87d7305594c5-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/d38dce7d1890ef61d80a87d7305594c5/family1 2023-11-28 03:55:22,858 DEBUG [StoreOpener-d38dce7d1890ef61d80a87d7305594c5-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/d38dce7d1890ef61d80a87d7305594c5/family1 2023-11-28 03:55:22,859 INFO [PEWorker-3] procedure2.ProcedureExecutor(1409): Finished pid=107, ppid=105, state=SUCCESS; TransitRegionStateProcedure table=ns:testMinorCompaction, region=227f7071ca3e7b6612f5a149eeb75e98, ASSIGN in 330 msec 2023-11-28 03:55:22,859 INFO [StoreOpener-d38dce7d1890ef61d80a87d7305594c5-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region d38dce7d1890ef61d80a87d7305594c5 columnFamilyName family1 2023-11-28 03:55:22,860 INFO [StoreOpener-d38dce7d1890ef61d80a87d7305594c5-1] regionserver.HStore(310): Store=d38dce7d1890ef61d80a87d7305594c5/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-28 03:55:22,860 INFO [StoreOpener-d38dce7d1890ef61d80a87d7305594c5-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region d38dce7d1890ef61d80a87d7305594c5 2023-11-28 03:55:22,861 DEBUG [StoreOpener-d38dce7d1890ef61d80a87d7305594c5-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/d38dce7d1890ef61d80a87d7305594c5/family2 2023-11-28 03:55:22,861 DEBUG [StoreOpener-d38dce7d1890ef61d80a87d7305594c5-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/d38dce7d1890ef61d80a87d7305594c5/family2 2023-11-28 03:55:22,861 INFO [StoreOpener-d38dce7d1890ef61d80a87d7305594c5-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region d38dce7d1890ef61d80a87d7305594c5 columnFamilyName family2 2023-11-28 03:55:22,862 INFO [StoreOpener-d38dce7d1890ef61d80a87d7305594c5-1] regionserver.HStore(310): Store=d38dce7d1890ef61d80a87d7305594c5/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-28 03:55:22,863 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/d38dce7d1890ef61d80a87d7305594c5 2023-11-28 03:55:22,863 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/d38dce7d1890ef61d80a87d7305594c5 2023-11-28 03:55:22,864 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table ns:testMinorCompaction descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-11-28 03:55:22,865 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for d38dce7d1890ef61d80a87d7305594c5 2023-11-28 03:55:22,869 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/d38dce7d1890ef61d80a87d7305594c5/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-11-28 03:55:22,869 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened d38dce7d1890ef61d80a87d7305594c5; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=70194134, jitterRate=0.04597410559654236}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-11-28 03:55:22,869 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for d38dce7d1890ef61d80a87d7305594c5: 2023-11-28 03:55:22,870 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2339): Post open deploy tasks for ns:testMinorCompaction,2,1701143722033.d38dce7d1890ef61d80a87d7305594c5., pid=110, masterSystemTime=1701143722836 2023-11-28 03:55:22,871 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2366): Finished post open deploy task for ns:testMinorCompaction,2,1701143722033.d38dce7d1890ef61d80a87d7305594c5. 2023-11-28 03:55:22,871 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened ns:testMinorCompaction,2,1701143722033.d38dce7d1890ef61d80a87d7305594c5. 2023-11-28 03:55:22,871 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open ns:testMinorCompaction,,1701143722033.8f7c6ece6778b5f8b473ef7daf18a3c4. 2023-11-28 03:55:22,871 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=108 updating hbase:meta row=d38dce7d1890ef61d80a87d7305594c5, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:22,871 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 8f7c6ece6778b5f8b473ef7daf18a3c4, NAME => 'ns:testMinorCompaction,,1701143722033.8f7c6ece6778b5f8b473ef7daf18a3c4.', STARTKEY => '', ENDKEY => '1'} 2023-11-28 03:55:22,871 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"ns:testMinorCompaction,2,1701143722033.d38dce7d1890ef61d80a87d7305594c5.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1701143722871"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1701143722871"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1701143722871"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1701143722871"}]},"ts":"1701143722871"} 2023-11-28 03:55:22,871 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMinorCompaction 8f7c6ece6778b5f8b473ef7daf18a3c4 2023-11-28 03:55:22,871 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated ns:testMinorCompaction,,1701143722033.8f7c6ece6778b5f8b473ef7daf18a3c4.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-28 03:55:22,872 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 8f7c6ece6778b5f8b473ef7daf18a3c4 2023-11-28 03:55:22,872 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 8f7c6ece6778b5f8b473ef7daf18a3c4 2023-11-28 03:55:22,873 INFO [StoreOpener-8f7c6ece6778b5f8b473ef7daf18a3c4-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 8f7c6ece6778b5f8b473ef7daf18a3c4 2023-11-28 03:55:22,874 INFO [PEWorker-4] procedure2.ProcedureExecutor(1825): Finished subprocedure pid=110, resume processing ppid=108 2023-11-28 03:55:22,874 INFO [PEWorker-4] procedure2.ProcedureExecutor(1409): Finished pid=110, ppid=108, state=SUCCESS; OpenRegionProcedure d38dce7d1890ef61d80a87d7305594c5, server=jenkins-hbase4.apache.org,41203,1701143600653 in 187 msec 2023-11-28 03:55:22,874 DEBUG [StoreOpener-8f7c6ece6778b5f8b473ef7daf18a3c4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/8f7c6ece6778b5f8b473ef7daf18a3c4/family1 2023-11-28 03:55:22,874 DEBUG [StoreOpener-8f7c6ece6778b5f8b473ef7daf18a3c4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/8f7c6ece6778b5f8b473ef7daf18a3c4/family1 2023-11-28 03:55:22,874 INFO [StoreOpener-8f7c6ece6778b5f8b473ef7daf18a3c4-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 8f7c6ece6778b5f8b473ef7daf18a3c4 columnFamilyName family1 2023-11-28 03:55:22,875 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=108, ppid=105, state=SUCCESS; TransitRegionStateProcedure table=ns:testMinorCompaction, region=d38dce7d1890ef61d80a87d7305594c5, ASSIGN in 346 msec 2023-11-28 03:55:22,875 INFO [StoreOpener-8f7c6ece6778b5f8b473ef7daf18a3c4-1] regionserver.HStore(310): Store=8f7c6ece6778b5f8b473ef7daf18a3c4/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-28 03:55:22,875 INFO [StoreOpener-8f7c6ece6778b5f8b473ef7daf18a3c4-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 8f7c6ece6778b5f8b473ef7daf18a3c4 2023-11-28 03:55:22,876 DEBUG [StoreOpener-8f7c6ece6778b5f8b473ef7daf18a3c4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/8f7c6ece6778b5f8b473ef7daf18a3c4/family2 2023-11-28 03:55:22,876 DEBUG [StoreOpener-8f7c6ece6778b5f8b473ef7daf18a3c4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/8f7c6ece6778b5f8b473ef7daf18a3c4/family2 2023-11-28 03:55:22,877 INFO [StoreOpener-8f7c6ece6778b5f8b473ef7daf18a3c4-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 8f7c6ece6778b5f8b473ef7daf18a3c4 columnFamilyName family2 2023-11-28 03:55:22,877 INFO [StoreOpener-8f7c6ece6778b5f8b473ef7daf18a3c4-1] regionserver.HStore(310): Store=8f7c6ece6778b5f8b473ef7daf18a3c4/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-28 03:55:22,878 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/8f7c6ece6778b5f8b473ef7daf18a3c4 2023-11-28 03:55:22,878 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/8f7c6ece6778b5f8b473ef7daf18a3c4 2023-11-28 03:55:22,880 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table ns:testMinorCompaction descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-11-28 03:55:22,881 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 8f7c6ece6778b5f8b473ef7daf18a3c4 2023-11-28 03:55:22,882 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/8f7c6ece6778b5f8b473ef7daf18a3c4/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-11-28 03:55:22,882 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 8f7c6ece6778b5f8b473ef7daf18a3c4; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=66945710, jitterRate=-0.0024311840534210205}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-11-28 03:55:22,882 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 8f7c6ece6778b5f8b473ef7daf18a3c4: 2023-11-28 03:55:22,883 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2339): Post open deploy tasks for ns:testMinorCompaction,,1701143722033.8f7c6ece6778b5f8b473ef7daf18a3c4., pid=111, masterSystemTime=1701143722836 2023-11-28 03:55:22,884 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2366): Finished post open deploy task for ns:testMinorCompaction,,1701143722033.8f7c6ece6778b5f8b473ef7daf18a3c4. 2023-11-28 03:55:22,884 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened ns:testMinorCompaction,,1701143722033.8f7c6ece6778b5f8b473ef7daf18a3c4. 2023-11-28 03:55:22,884 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=106 updating hbase:meta row=8f7c6ece6778b5f8b473ef7daf18a3c4, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:22,884 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"ns:testMinorCompaction,,1701143722033.8f7c6ece6778b5f8b473ef7daf18a3c4.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1701143722884"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1701143722884"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1701143722884"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1701143722884"}]},"ts":"1701143722884"} 2023-11-28 03:55:22,886 INFO [PEWorker-5] procedure2.ProcedureExecutor(1825): Finished subprocedure pid=111, resume processing ppid=106 2023-11-28 03:55:22,886 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=111, ppid=106, state=SUCCESS; OpenRegionProcedure 8f7c6ece6778b5f8b473ef7daf18a3c4, server=jenkins-hbase4.apache.org,41203,1701143600653 in 200 msec 2023-11-28 03:55:22,887 INFO [PEWorker-2] procedure2.ProcedureExecutor(1825): Finished subprocedure pid=106, resume processing ppid=105 2023-11-28 03:55:22,887 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=106, ppid=105, state=SUCCESS; TransitRegionStateProcedure table=ns:testMinorCompaction, region=8f7c6ece6778b5f8b473ef7daf18a3c4, ASSIGN in 358 msec 2023-11-28 03:55:22,888 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-11-28 03:55:22,888 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"ns:testMinorCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1701143722888"}]},"ts":"1701143722888"} 2023-11-28 03:55:22,889 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=ns:testMinorCompaction, state=ENABLED in hbase:meta 2023-11-28 03:55:22,891 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_POST_OPERATION 2023-11-28 03:55:22,892 INFO [PEWorker-3] procedure2.ProcedureExecutor(1409): Finished pid=105, state=SUCCESS; CreateTableProcedure table=ns:testMinorCompaction in 857 msec 2023-11-28 03:55:22,905 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family1/testCompactionWithHFileLink=b3c2fdb68f55a3cbe8d29ee7b6ddd5e5-88a76a3cbca146b6bb71214ecc4a9c87, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family1/testCompactionWithHFileLink=b3c2fdb68f55a3cbe8d29ee7b6ddd5e5-299d98e37656434ea5b2482137c58710, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family1/testCompactionWithHFileLink=b3c2fdb68f55a3cbe8d29ee7b6ddd5e5-dc7b41ef849648dda6e80bfb0cdeea0f, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family1/testCompactionWithHFileLink=b3c2fdb68f55a3cbe8d29ee7b6ddd5e5-34bdbacde3fd4331ae0cc21041140fd9, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family1/f34c57f507ce46a8bf8c46a25c5472ca_SeqId_34_] to archive 2023-11-28 03:55:22,906 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-28 03:55:22,907 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family1/testCompactionWithHFileLink=b3c2fdb68f55a3cbe8d29ee7b6ddd5e5-88a76a3cbca146b6bb71214ecc4a9c87 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family1/testCompactionWithHFileLink=b3c2fdb68f55a3cbe8d29ee7b6ddd5e5-88a76a3cbca146b6bb71214ecc4a9c87 2023-11-28 03:55:22,908 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family1/testCompactionWithHFileLink=b3c2fdb68f55a3cbe8d29ee7b6ddd5e5-299d98e37656434ea5b2482137c58710 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family1/testCompactionWithHFileLink=b3c2fdb68f55a3cbe8d29ee7b6ddd5e5-299d98e37656434ea5b2482137c58710 2023-11-28 03:55:22,909 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family1/testCompactionWithHFileLink=b3c2fdb68f55a3cbe8d29ee7b6ddd5e5-dc7b41ef849648dda6e80bfb0cdeea0f to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family1/testCompactionWithHFileLink=b3c2fdb68f55a3cbe8d29ee7b6ddd5e5-dc7b41ef849648dda6e80bfb0cdeea0f 2023-11-28 03:55:22,910 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family1/testCompactionWithHFileLink=b3c2fdb68f55a3cbe8d29ee7b6ddd5e5-34bdbacde3fd4331ae0cc21041140fd9 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family1/testCompactionWithHFileLink=b3c2fdb68f55a3cbe8d29ee7b6ddd5e5-34bdbacde3fd4331ae0cc21041140fd9 2023-11-28 03:55:22,911 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family1/f34c57f507ce46a8bf8c46a25c5472ca_SeqId_34_ to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/family1/f34c57f507ce46a8bf8c46a25c5472ca_SeqId_34_ 2023-11-28 03:55:22,913 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family2/testCompactionWithHFileLink=2aad05dc9c73dcbea66259e5c0a6958f-a79a53b7062d4e53b9199b02c23f6c24, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family2/testCompactionWithHFileLink=2aad05dc9c73dcbea66259e5c0a6958f-a5d2b1b5f59d4745beda10e8ec8e729f, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family2/testCompactionWithHFileLink=2aad05dc9c73dcbea66259e5c0a6958f-9d803becce0f4ac4ad8171434ff9b0a2, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family2/testCompactionWithHFileLink=2aad05dc9c73dcbea66259e5c0a6958f-cdc6a50b83e147878684e9d41c01c3e6] to archive 2023-11-28 03:55:22,913 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-28 03:55:22,914 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family2/testCompactionWithHFileLink=2aad05dc9c73dcbea66259e5c0a6958f-a79a53b7062d4e53b9199b02c23f6c24 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family2/testCompactionWithHFileLink=2aad05dc9c73dcbea66259e5c0a6958f-a79a53b7062d4e53b9199b02c23f6c24 2023-11-28 03:55:22,915 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family2/testCompactionWithHFileLink=2aad05dc9c73dcbea66259e5c0a6958f-a5d2b1b5f59d4745beda10e8ec8e729f to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family2/testCompactionWithHFileLink=2aad05dc9c73dcbea66259e5c0a6958f-a5d2b1b5f59d4745beda10e8ec8e729f 2023-11-28 03:55:22,916 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family2/testCompactionWithHFileLink=2aad05dc9c73dcbea66259e5c0a6958f-9d803becce0f4ac4ad8171434ff9b0a2 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family2/testCompactionWithHFileLink=2aad05dc9c73dcbea66259e5c0a6958f-9d803becce0f4ac4ad8171434ff9b0a2 2023-11-28 03:55:22,917 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family2/testCompactionWithHFileLink=2aad05dc9c73dcbea66259e5c0a6958f-cdc6a50b83e147878684e9d41c01c3e6 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/family2/testCompactionWithHFileLink=2aad05dc9c73dcbea66259e5c0a6958f-cdc6a50b83e147878684e9d41c01c3e6 2023-11-28 03:55:23,142 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] master.MasterRpcServices(1230): Checking to see if procedure is done pid=105 2023-11-28 03:55:23,142 INFO [Listener at localhost/42853] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: ns:testMinorCompaction, procId: 105 completed 2023-11-28 03:55:23,148 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41203] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,,1701143722033.8f7c6ece6778b5f8b473ef7daf18a3c4. with WAL disabled. Data may be lost in the event of a crash. 2023-11-28 03:55:23,149 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41203] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,1,1701143722033.227f7071ca3e7b6612f5a149eeb75e98. with WAL disabled. Data may be lost in the event of a crash. 2023-11-28 03:55:23,149 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41203] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,2,1701143722033.d38dce7d1890ef61d80a87d7305594c5. with WAL disabled. Data may be lost in the event of a crash. 2023-11-28 03:55:23,151 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-28 03:55:23,153 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] procedure.ProcedureCoordinator(165): Submitting procedure ns:testMinorCompaction 2023-11-28 03:55:23,153 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns:testMinorCompaction' 2023-11-28 03:55:23,153 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:55:23,153 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns:testMinorCompaction' starting 'acquire' 2023-11-28 03:55:23,153 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns:testMinorCompaction', kicking off acquire phase on members. 2023-11-28 03:55:23,153 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-28 03:55:23,153 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-11-28 03:55:23,155 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:55:23,155 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:23,155 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:55:23,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:55:23,155 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:23,155 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-28 03:55:23,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-11-28 03:55:23,155 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-28 03:55:23,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-28 03:55:23,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-11-28 03:55:23,156 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns:testMinorCompaction 2023-11-28 03:55:23,156 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns:testMinorCompaction 2023-11-28 03:55:23,156 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns:testMinorCompaction' with timeout 60000ms 2023-11-28 03:55:23,156 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:55:23,156 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns:testMinorCompaction' starting 'acquire' stage 2023-11-28 03:55:23,157 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,1,1701143722033.227f7071ca3e7b6612f5a149eeb75e98. 2023-11-28 03:55:23,157 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,1,1701143722033.227f7071ca3e7b6612f5a149eeb75e98. started... 2023-11-28 03:55:23,157 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,,1701143722033.8f7c6ece6778b5f8b473ef7daf18a3c4. 2023-11-28 03:55:23,158 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,,1701143722033.8f7c6ece6778b5f8b473ef7daf18a3c4. started... 2023-11-28 03:55:23,158 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,2,1701143722033.d38dce7d1890ef61d80a87d7305594c5. 2023-11-28 03:55:23,157 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-11-28 03:55:23,158 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 8f7c6ece6778b5f8b473ef7daf18a3c4 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-11-28 03:55:23,158 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,2,1701143722033.d38dce7d1890ef61d80a87d7305594c5. started... 2023-11-28 03:55:23,158 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 227f7071ca3e7b6612f5a149eeb75e98 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-11-28 03:55:23,158 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-28 03:55:23,159 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing d38dce7d1890ef61d80a87d7305594c5 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-11-28 03:55:23,579 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023112865a3d186c99a4a8cbb01d7ce281c6a97 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e2023112865a3d186c99a4a8cbb01d7ce281c6a97 2023-11-28 03:55:23,579 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202311287e09242846d741afb55ff45c59b2dc73 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b202311287e09242846d741afb55ff45c59b2dc73 2023-11-28 03:55:23,585 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202311284c388f1af8be4ade99ec99cac0274a00 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c202311284c388f1af8be4ade99ec99cac0274a00 2023-11-28 03:55:23,590 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/d38dce7d1890ef61d80a87d7305594c5/.tmp/family1/0913518fbcf6490eac43fd7fdc96efab 2023-11-28 03:55:23,602 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202311284a47d39894414130aba63e9eff22a011 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c81e728d9d4c2f636f067f89cc14862c202311284a47d39894414130aba63e9eff22a011 2023-11-28 03:55:23,985 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/8f7c6ece6778b5f8b473ef7daf18a3c4/.tmp/family1/f8b2846f7d5e4343b60b0ae9a79f9dba 2023-11-28 03:55:23,985 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/227f7071ca3e7b6612f5a149eeb75e98/.tmp/family1/67113a419a8e409eb5707edeebe4c7e9 2023-11-28 03:55:23,998 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202311281ea351e7c4fd428ba05801ff4dda4677 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c4ca4238a0b923820dcc509a6f75849b202311281ea351e7c4fd428ba05801ff4dda4677 2023-11-28 03:55:24,006 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/d38dce7d1890ef61d80a87d7305594c5/.tmp/family2/844d4831eabb45148613e543e3187b99 2023-11-28 03:55:24,010 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/d38dce7d1890ef61d80a87d7305594c5/.tmp/family1/0913518fbcf6490eac43fd7fdc96efab as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/d38dce7d1890ef61d80a87d7305594c5/family1/0913518fbcf6490eac43fd7fdc96efab 2023-11-28 03:55:24,013 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/d38dce7d1890ef61d80a87d7305594c5/family1/0913518fbcf6490eac43fd7fdc96efab, entries=4, sequenceid=5, filesize=5.4 K 2023-11-28 03:55:24,013 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/d38dce7d1890ef61d80a87d7305594c5/.tmp/family2/844d4831eabb45148613e543e3187b99 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/d38dce7d1890ef61d80a87d7305594c5/family2/844d4831eabb45148613e543e3187b99 2023-11-28 03:55:24,016 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/d38dce7d1890ef61d80a87d7305594c5/family2/844d4831eabb45148613e543e3187b99, entries=2, sequenceid=5, filesize=5.1 K 2023-11-28 03:55:24,017 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for d38dce7d1890ef61d80a87d7305594c5 in 858ms, sequenceid=5, compaction requested=false 2023-11-28 03:55:24,017 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'ns:testMinorCompaction' 2023-11-28 03:55:24,018 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for d38dce7d1890ef61d80a87d7305594c5: 2023-11-28 03:55:24,018 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,2,1701143722033.d38dce7d1890ef61d80a87d7305594c5. 2023-11-28 03:55:24,018 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-11-28 03:55:24,399 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202311283b5a7ba6610b4d4d805372e7058c846d to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/d41d8cd98f00b204e9800998ecf8427e202311283b5a7ba6610b4d4d805372e7058c846d 2023-11-28 03:55:24,402 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/227f7071ca3e7b6612f5a149eeb75e98/.tmp/family2/fbaad07b4ade42ca9a0cb454b0a3170a 2023-11-28 03:55:24,406 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/227f7071ca3e7b6612f5a149eeb75e98/.tmp/family1/67113a419a8e409eb5707edeebe4c7e9 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/227f7071ca3e7b6612f5a149eeb75e98/family1/67113a419a8e409eb5707edeebe4c7e9 2023-11-28 03:55:24,409 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/227f7071ca3e7b6612f5a149eeb75e98/family1/67113a419a8e409eb5707edeebe4c7e9, entries=4, sequenceid=5, filesize=5.4 K 2023-11-28 03:55:24,410 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/227f7071ca3e7b6612f5a149eeb75e98/.tmp/family2/fbaad07b4ade42ca9a0cb454b0a3170a as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/227f7071ca3e7b6612f5a149eeb75e98/family2/fbaad07b4ade42ca9a0cb454b0a3170a 2023-11-28 03:55:24,414 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/227f7071ca3e7b6612f5a149eeb75e98/family2/fbaad07b4ade42ca9a0cb454b0a3170a, entries=2, sequenceid=5, filesize=5.1 K 2023-11-28 03:55:24,414 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 227f7071ca3e7b6612f5a149eeb75e98 in 1256ms, sequenceid=5, compaction requested=false 2023-11-28 03:55:24,415 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 227f7071ca3e7b6612f5a149eeb75e98: 2023-11-28 03:55:24,415 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,1,1701143722033.227f7071ca3e7b6612f5a149eeb75e98. 2023-11-28 03:55:24,415 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-11-28 03:55:24,804 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/8f7c6ece6778b5f8b473ef7daf18a3c4/.tmp/family2/068d7e9e4de34c42be56a2ee651a2882 2023-11-28 03:55:24,809 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/8f7c6ece6778b5f8b473ef7daf18a3c4/.tmp/family1/f8b2846f7d5e4343b60b0ae9a79f9dba as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/8f7c6ece6778b5f8b473ef7daf18a3c4/family1/f8b2846f7d5e4343b60b0ae9a79f9dba 2023-11-28 03:55:24,813 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/8f7c6ece6778b5f8b473ef7daf18a3c4/family1/f8b2846f7d5e4343b60b0ae9a79f9dba, entries=4, sequenceid=5, filesize=5.4 K 2023-11-28 03:55:24,814 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/8f7c6ece6778b5f8b473ef7daf18a3c4/.tmp/family2/068d7e9e4de34c42be56a2ee651a2882 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/8f7c6ece6778b5f8b473ef7daf18a3c4/family2/068d7e9e4de34c42be56a2ee651a2882 2023-11-28 03:55:24,817 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/8f7c6ece6778b5f8b473ef7daf18a3c4/family2/068d7e9e4de34c42be56a2ee651a2882, entries=2, sequenceid=5, filesize=5.1 K 2023-11-28 03:55:24,818 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 8f7c6ece6778b5f8b473ef7daf18a3c4 in 1660ms, sequenceid=5, compaction requested=false 2023-11-28 03:55:24,818 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 8f7c6ece6778b5f8b473ef7daf18a3c4: 2023-11-28 03:55:24,819 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,,1701143722033.8f7c6ece6778b5f8b473ef7daf18a3c4. 2023-11-28 03:55:24,819 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-11-28 03:55:24,819 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-11-28 03:55:24,819 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:24,819 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns:testMinorCompaction' locally acquired 2023-11-28 03:55:24,819 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure (ns:testMinorCompaction) in zk 2023-11-28 03:55:24,831 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-28 03:55:24,831 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:24,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:24,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:55:24,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:55:24,832 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-28 03:55:24,832 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns:testMinorCompaction' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-28 03:55:24,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:55:24,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:55:24,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-11-28 03:55:24,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:24,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:55:24,834 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure 'ns:testMinorCompaction' on coordinator 2023-11-28 03:55:24,834 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@14484b2e[Count = 0] remaining members to acquire global barrier 2023-11-28 03:55:24,834 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns:testMinorCompaction' starting 'in-barrier' execution. 2023-11-28 03:55:24,834 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-28 03:55:24,846 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-28 03:55:24,846 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-28 03:55:24,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-28 03:55:24,846 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns:testMinorCompaction' received 'reached' from coordinator. 2023-11-28 03:55:24,846 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns:testMinorCompaction' locally completed 2023-11-28 03:55:24,846 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns:testMinorCompaction' completed for member 'jenkins-hbase4.apache.org,41203,1701143600653' in zk 2023-11-28 03:55:24,846 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:24,846 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-28 03:55:24,852 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:24,852 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns:testMinorCompaction' has notified controller of completion 2023-11-28 03:55:24,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:24,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:55:24,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:55:24,852 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:55:24,852 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns:testMinorCompaction' completed. 2023-11-28 03:55:24,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:55:24,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:55:24,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-11-28 03:55:24,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:24,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:55:24,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-11-28 03:55:24,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:24,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'ns:testMinorCompaction' member 'jenkins-hbase4.apache.org,41203,1701143600653': 2023-11-28 03:55:24,857 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' released barrier for procedure'ns:testMinorCompaction', counting down latch. Waiting for 0 more 2023-11-28 03:55:24,857 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns:testMinorCompaction' execution completed 2023-11-28 03:55:24,857 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-28 03:55:24,857 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-28 03:55:24,857 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns:testMinorCompaction 2023-11-28 03:55:24,857 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns:testMinorCompactionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-28 03:55:24,860 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-28 03:55:24,860 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-28 03:55:24,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-28 03:55:24,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:55:24,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:55:24,861 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-28 03:55:24,860 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:55:24,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-28 03:55:24,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:55:24,861 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:55:24,861 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:24,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:55:24,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-11-28 03:55:24,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-28 03:55:24,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:55:24,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-11-28 03:55:24,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:24,863 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:24,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:55:24,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-11-28 03:55:24,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:24,865 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:24,865 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:55:24,865 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-11-28 03:55:24,866 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns:testMinorCompaction' 2023-11-28 03:55:24,866 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:55:24,866 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:55:24,866 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:55:24,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:55:24,866 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-28 03:55:24,866 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-11-28 03:55:24,866 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:24,866 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-28 03:55:24,866 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-28 03:55:24,866 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-28 03:55:24,866 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:55:24,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:55:24,866 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns:testMinorCompaction'' to complete. (max 300000 ms per retry) 2023-11-28 03:55:24,866 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-28 03:55:24,966 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-28 03:55:24,967 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-28 03:55:24,969 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41203] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,,1701143722033.8f7c6ece6778b5f8b473ef7daf18a3c4. with WAL disabled. Data may be lost in the event of a crash. 2023-11-28 03:55:24,969 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41203] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,1,1701143722033.227f7071ca3e7b6612f5a149eeb75e98. with WAL disabled. Data may be lost in the event of a crash. 2023-11-28 03:55:24,970 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41203] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,2,1701143722033.d38dce7d1890ef61d80a87d7305594c5. with WAL disabled. Data may be lost in the event of a crash. 2023-11-28 03:55:24,972 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-28 03:55:24,973 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] procedure.ProcedureCoordinator(143): Procedure ns:testMinorCompaction was in running list but was completed. Accepting new attempt. 2023-11-28 03:55:24,973 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] procedure.ProcedureCoordinator(165): Submitting procedure ns:testMinorCompaction 2023-11-28 03:55:24,973 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns:testMinorCompaction' 2023-11-28 03:55:24,973 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:55:24,973 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns:testMinorCompaction' starting 'acquire' 2023-11-28 03:55:24,973 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns:testMinorCompaction', kicking off acquire phase on members. 2023-11-28 03:55:24,974 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-28 03:55:24,974 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-11-28 03:55:24,975 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:24,975 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:55:24,976 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:55:24,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:55:24,976 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:24,976 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-28 03:55:24,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-11-28 03:55:24,976 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-28 03:55:24,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-28 03:55:24,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-11-28 03:55:24,977 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns:testMinorCompaction 2023-11-28 03:55:24,980 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc ns:testMinorCompaction is still present, removing 2023-11-28 03:55:24,980 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns:testMinorCompaction 2023-11-28 03:55:24,980 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns:testMinorCompaction' with timeout 60000ms 2023-11-28 03:55:24,980 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:55:24,980 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns:testMinorCompaction' starting 'acquire' stage 2023-11-28 03:55:24,980 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,1,1701143722033.227f7071ca3e7b6612f5a149eeb75e98. 2023-11-28 03:55:24,980 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,1,1701143722033.227f7071ca3e7b6612f5a149eeb75e98. started... 2023-11-28 03:55:24,980 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,,1701143722033.8f7c6ece6778b5f8b473ef7daf18a3c4. 2023-11-28 03:55:24,980 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,,1701143722033.8f7c6ece6778b5f8b473ef7daf18a3c4. started... 2023-11-28 03:55:24,980 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 227f7071ca3e7b6612f5a149eeb75e98 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-11-28 03:55:24,980 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-11-28 03:55:24,981 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-28 03:55:24,980 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,2,1701143722033.d38dce7d1890ef61d80a87d7305594c5. 2023-11-28 03:55:24,981 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,2,1701143722033.d38dce7d1890ef61d80a87d7305594c5. started... 2023-11-28 03:55:24,980 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 8f7c6ece6778b5f8b473ef7daf18a3c4 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-11-28 03:55:24,981 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing d38dce7d1890ef61d80a87d7305594c5 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-11-28 03:55:24,998 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023112865ce4f9d3f674f42b041279f78d28dcc to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e2023112865ce4f9d3f674f42b041279f78d28dcc 2023-11-28 03:55:24,999 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20231128ceea9ed07aef46c18d15ebf8fe3e0db5 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c20231128ceea9ed07aef46c18d15ebf8fe3e0db5 2023-11-28 03:55:24,999 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202311289d24824af9d14d9c9a9e38c06bed2523 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b202311289d24824af9d14d9c9a9e38c06bed2523 2023-11-28 03:55:25,012 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/d38dce7d1890ef61d80a87d7305594c5/.tmp/family1/3ca3b28c49a54775b7a01e8c930bfd85 2023-11-28 03:55:25,012 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/8f7c6ece6778b5f8b473ef7daf18a3c4/.tmp/family1/743e80d2d5f34d2b9a48382b09672de3 2023-11-28 03:55:25,034 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20231128d6dff76f09ef4fb087b6305fbc1079d4 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c81e728d9d4c2f636f067f89cc14862c20231128d6dff76f09ef4fb087b6305fbc1079d4 2023-11-28 03:55:25,417 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/227f7071ca3e7b6612f5a149eeb75e98/.tmp/family1/47d1fcd8025c4bd681fca058ee3dbec4 2023-11-28 03:55:25,434 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20231128ce1f6c8040e7449aad92976eba50aae3 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/d41d8cd98f00b204e9800998ecf8427e20231128ce1f6c8040e7449aad92976eba50aae3 2023-11-28 03:55:25,440 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/d38dce7d1890ef61d80a87d7305594c5/.tmp/family2/75dc829189a8422093cdb3bd57770ede 2023-11-28 03:55:25,444 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/d38dce7d1890ef61d80a87d7305594c5/.tmp/family1/3ca3b28c49a54775b7a01e8c930bfd85 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/d38dce7d1890ef61d80a87d7305594c5/family1/3ca3b28c49a54775b7a01e8c930bfd85 2023-11-28 03:55:25,448 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/d38dce7d1890ef61d80a87d7305594c5/family1/3ca3b28c49a54775b7a01e8c930bfd85, entries=4, sequenceid=9, filesize=5.4 K 2023-11-28 03:55:25,448 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/d38dce7d1890ef61d80a87d7305594c5/.tmp/family2/75dc829189a8422093cdb3bd57770ede as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/d38dce7d1890ef61d80a87d7305594c5/family2/75dc829189a8422093cdb3bd57770ede 2023-11-28 03:55:25,451 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/d38dce7d1890ef61d80a87d7305594c5/family2/75dc829189a8422093cdb3bd57770ede, entries=2, sequenceid=9, filesize=5.1 K 2023-11-28 03:55:25,452 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for d38dce7d1890ef61d80a87d7305594c5 in 471ms, sequenceid=9, compaction requested=false 2023-11-28 03:55:25,452 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for d38dce7d1890ef61d80a87d7305594c5: 2023-11-28 03:55:25,452 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,2,1701143722033.d38dce7d1890ef61d80a87d7305594c5. 2023-11-28 03:55:25,452 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-11-28 03:55:25,830 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202311280c0ef2f5a53644ae90dc84b7b64f42d3 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c4ca4238a0b923820dcc509a6f75849b202311280c0ef2f5a53644ae90dc84b7b64f42d3 2023-11-28 03:55:25,835 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/227f7071ca3e7b6612f5a149eeb75e98/.tmp/family2/71d360d7554c4350a5974ded5d9257b0 2023-11-28 03:55:25,839 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/8f7c6ece6778b5f8b473ef7daf18a3c4/.tmp/family2/50b8f1072a5f40d4a3264d767aedb922 2023-11-28 03:55:25,840 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/227f7071ca3e7b6612f5a149eeb75e98/.tmp/family1/47d1fcd8025c4bd681fca058ee3dbec4 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/227f7071ca3e7b6612f5a149eeb75e98/family1/47d1fcd8025c4bd681fca058ee3dbec4 2023-11-28 03:55:25,844 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/8f7c6ece6778b5f8b473ef7daf18a3c4/.tmp/family1/743e80d2d5f34d2b9a48382b09672de3 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/8f7c6ece6778b5f8b473ef7daf18a3c4/family1/743e80d2d5f34d2b9a48382b09672de3 2023-11-28 03:55:25,844 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/227f7071ca3e7b6612f5a149eeb75e98/family1/47d1fcd8025c4bd681fca058ee3dbec4, entries=4, sequenceid=9, filesize=5.4 K 2023-11-28 03:55:25,845 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/227f7071ca3e7b6612f5a149eeb75e98/.tmp/family2/71d360d7554c4350a5974ded5d9257b0 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/227f7071ca3e7b6612f5a149eeb75e98/family2/71d360d7554c4350a5974ded5d9257b0 2023-11-28 03:55:25,848 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/8f7c6ece6778b5f8b473ef7daf18a3c4/family1/743e80d2d5f34d2b9a48382b09672de3, entries=4, sequenceid=9, filesize=5.4 K 2023-11-28 03:55:25,849 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/8f7c6ece6778b5f8b473ef7daf18a3c4/.tmp/family2/50b8f1072a5f40d4a3264d767aedb922 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/8f7c6ece6778b5f8b473ef7daf18a3c4/family2/50b8f1072a5f40d4a3264d767aedb922 2023-11-28 03:55:25,850 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/227f7071ca3e7b6612f5a149eeb75e98/family2/71d360d7554c4350a5974ded5d9257b0, entries=2, sequenceid=9, filesize=5.1 K 2023-11-28 03:55:25,851 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 227f7071ca3e7b6612f5a149eeb75e98 in 871ms, sequenceid=9, compaction requested=false 2023-11-28 03:55:25,851 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 227f7071ca3e7b6612f5a149eeb75e98: 2023-11-28 03:55:25,851 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,1,1701143722033.227f7071ca3e7b6612f5a149eeb75e98. 2023-11-28 03:55:25,851 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-11-28 03:55:25,853 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/8f7c6ece6778b5f8b473ef7daf18a3c4/family2/50b8f1072a5f40d4a3264d767aedb922, entries=2, sequenceid=9, filesize=5.1 K 2023-11-28 03:55:25,854 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 8f7c6ece6778b5f8b473ef7daf18a3c4 in 874ms, sequenceid=9, compaction requested=false 2023-11-28 03:55:25,854 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 8f7c6ece6778b5f8b473ef7daf18a3c4: 2023-11-28 03:55:25,854 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,,1701143722033.8f7c6ece6778b5f8b473ef7daf18a3c4. 2023-11-28 03:55:25,854 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-11-28 03:55:25,855 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-11-28 03:55:25,855 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:25,855 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns:testMinorCompaction' locally acquired 2023-11-28 03:55:25,855 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure (ns:testMinorCompaction) in zk 2023-11-28 03:55:25,858 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:25,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:25,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:55:25,858 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-28 03:55:25,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:55:25,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:55:25,858 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-28 03:55:25,858 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns:testMinorCompaction' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-28 03:55:25,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:55:25,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-11-28 03:55:25,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:25,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:55:25,860 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure 'ns:testMinorCompaction' on coordinator 2023-11-28 03:55:25,860 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1de7b5e4[Count = 0] remaining members to acquire global barrier 2023-11-28 03:55:25,860 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns:testMinorCompaction' starting 'in-barrier' execution. 2023-11-28 03:55:25,860 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-28 03:55:25,861 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-28 03:55:25,861 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-28 03:55:25,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-28 03:55:25,861 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns:testMinorCompaction' received 'reached' from coordinator. 2023-11-28 03:55:25,861 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns:testMinorCompaction' locally completed 2023-11-28 03:55:25,861 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns:testMinorCompaction' completed for member 'jenkins-hbase4.apache.org,41203,1701143600653' in zk 2023-11-28 03:55:25,862 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:25,862 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-28 03:55:25,863 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:25,863 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns:testMinorCompaction' has notified controller of completion 2023-11-28 03:55:25,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:25,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:55:25,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:55:25,863 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:55:25,864 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns:testMinorCompaction' completed. 2023-11-28 03:55:25,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:55:25,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:55:25,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-11-28 03:55:25,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:25,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:55:25,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-11-28 03:55:25,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:25,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'ns:testMinorCompaction' member 'jenkins-hbase4.apache.org,41203,1701143600653': 2023-11-28 03:55:25,867 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' released barrier for procedure'ns:testMinorCompaction', counting down latch. Waiting for 0 more 2023-11-28 03:55:25,867 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns:testMinorCompaction' execution completed 2023-11-28 03:55:25,867 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-28 03:55:25,867 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-28 03:55:25,867 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns:testMinorCompaction 2023-11-28 03:55:25,867 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns:testMinorCompactionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-28 03:55:25,869 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-28 03:55:25,869 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-28 03:55:25,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-28 03:55:25,869 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-28 03:55:25,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:55:25,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:55:25,869 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:55:25,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-28 03:55:25,870 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:55:25,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:55:25,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:55:25,870 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:25,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-28 03:55:25,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-11-28 03:55:25,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:55:25,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-11-28 03:55:25,871 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:25,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:25,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:55:25,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-11-28 03:55:25,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:25,875 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:55:25,875 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:55:25,875 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:55:25,875 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:25,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:55:25,875 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns:testMinorCompaction' 2023-11-28 03:55:25,875 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-28 03:55:25,875 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:55:25,875 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-11-28 03:55:25,875 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-11-28 03:55:25,876 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:25,876 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:55:25,876 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns:testMinorCompaction'' to complete. (max 300000 ms per retry) 2023-11-28 03:55:25,876 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-28 03:55:25,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:55:25,876 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-28 03:55:25,876 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(399): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/ns:testMinorCompaction because node does not exist (not an error) 2023-11-28 03:55:25,876 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-28 03:55:25,876 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-28 03:55:25,976 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-28 03:55:25,977 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-28 03:55:25,978 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41203] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,,1701143722033.8f7c6ece6778b5f8b473ef7daf18a3c4. with WAL disabled. Data may be lost in the event of a crash. 2023-11-28 03:55:25,978 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41203] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,1,1701143722033.227f7071ca3e7b6612f5a149eeb75e98. with WAL disabled. Data may be lost in the event of a crash. 2023-11-28 03:55:25,978 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41203] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,2,1701143722033.d38dce7d1890ef61d80a87d7305594c5. with WAL disabled. Data may be lost in the event of a crash. 2023-11-28 03:55:25,981 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-28 03:55:25,982 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] procedure.ProcedureCoordinator(143): Procedure ns:testMinorCompaction was in running list but was completed. Accepting new attempt. 2023-11-28 03:55:25,982 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] procedure.ProcedureCoordinator(165): Submitting procedure ns:testMinorCompaction 2023-11-28 03:55:25,983 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns:testMinorCompaction' 2023-11-28 03:55:25,983 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:55:25,983 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns:testMinorCompaction' starting 'acquire' 2023-11-28 03:55:25,983 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns:testMinorCompaction', kicking off acquire phase on members. 2023-11-28 03:55:25,983 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-28 03:55:25,983 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-11-28 03:55:25,985 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:55:25,985 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:25,985 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:55:25,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:55:25,985 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:25,985 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-28 03:55:25,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-11-28 03:55:25,985 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-28 03:55:25,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-28 03:55:25,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-11-28 03:55:25,986 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns:testMinorCompaction 2023-11-28 03:55:25,986 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc ns:testMinorCompaction is still present, removing 2023-11-28 03:55:25,986 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns:testMinorCompaction 2023-11-28 03:55:25,986 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns:testMinorCompaction' with timeout 60000ms 2023-11-28 03:55:25,986 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:55:25,986 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns:testMinorCompaction' starting 'acquire' stage 2023-11-28 03:55:25,986 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,1,1701143722033.227f7071ca3e7b6612f5a149eeb75e98. 2023-11-28 03:55:25,987 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,1,1701143722033.227f7071ca3e7b6612f5a149eeb75e98. started... 2023-11-28 03:55:25,987 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 227f7071ca3e7b6612f5a149eeb75e98 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-11-28 03:55:25,988 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,,1701143722033.8f7c6ece6778b5f8b473ef7daf18a3c4. 2023-11-28 03:55:25,988 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-11-28 03:55:25,988 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-28 03:55:25,988 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,2,1701143722033.d38dce7d1890ef61d80a87d7305594c5. 2023-11-28 03:55:25,988 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,2,1701143722033.d38dce7d1890ef61d80a87d7305594c5. started... 2023-11-28 03:55:25,988 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,,1701143722033.8f7c6ece6778b5f8b473ef7daf18a3c4. started... 2023-11-28 03:55:25,988 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing d38dce7d1890ef61d80a87d7305594c5 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-11-28 03:55:25,988 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 8f7c6ece6778b5f8b473ef7daf18a3c4 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-11-28 03:55:26,014 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202311283de8cfb67c194ed7abde8385f32b7b5f to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c202311283de8cfb67c194ed7abde8385f32b7b5f 2023-11-28 03:55:26,015 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202311283a8d4d7ad342420d94e404710c342c04 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b202311283a8d4d7ad342420d94e404710c342c04 2023-11-28 03:55:26,015 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20231128466ea94fe9d643549f81169e9d5d5d1a to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e20231128466ea94fe9d643549f81169e9d5d5d1a 2023-11-28 03:55:26,030 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/d38dce7d1890ef61d80a87d7305594c5/.tmp/family1/32a8c14c2722489bb9a27951bdfa2146 2023-11-28 03:55:26,030 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/8f7c6ece6778b5f8b473ef7daf18a3c4/.tmp/family1/2bc75edf0b2e4e57916ebe8f6947fc32 2023-11-28 03:55:26,424 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/227f7071ca3e7b6612f5a149eeb75e98/.tmp/family1/07fcc1ac132145d298a6f5bf416de5b0 2023-11-28 03:55:26,453 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20231128ae320ab299da46f2aa13153c62be4e75 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/d41d8cd98f00b204e9800998ecf8427e20231128ae320ab299da46f2aa13153c62be4e75 2023-11-28 03:55:26,453 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202311285dbf87c3bc4e4cca9e176fac74f889fd to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c81e728d9d4c2f636f067f89cc14862c202311285dbf87c3bc4e4cca9e176fac74f889fd 2023-11-28 03:55:26,838 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20231128ab903d4ebc8c4485ae97c080711872aa to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c4ca4238a0b923820dcc509a6f75849b20231128ab903d4ebc8c4485ae97c080711872aa 2023-11-28 03:55:26,858 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/d38dce7d1890ef61d80a87d7305594c5/.tmp/family2/152c70cd7cc74b44a1fdad7c99339763 2023-11-28 03:55:26,858 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/8f7c6ece6778b5f8b473ef7daf18a3c4/.tmp/family2/3a331bbd4c8547858f2b52ac47ada923 2023-11-28 03:55:26,862 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/d38dce7d1890ef61d80a87d7305594c5/.tmp/family1/32a8c14c2722489bb9a27951bdfa2146 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/d38dce7d1890ef61d80a87d7305594c5/family1/32a8c14c2722489bb9a27951bdfa2146 2023-11-28 03:55:26,863 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/8f7c6ece6778b5f8b473ef7daf18a3c4/.tmp/family1/2bc75edf0b2e4e57916ebe8f6947fc32 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/8f7c6ece6778b5f8b473ef7daf18a3c4/family1/2bc75edf0b2e4e57916ebe8f6947fc32 2023-11-28 03:55:26,867 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/d38dce7d1890ef61d80a87d7305594c5/family1/32a8c14c2722489bb9a27951bdfa2146, entries=4, sequenceid=13, filesize=5.4 K 2023-11-28 03:55:26,867 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/8f7c6ece6778b5f8b473ef7daf18a3c4/family1/2bc75edf0b2e4e57916ebe8f6947fc32, entries=4, sequenceid=13, filesize=5.4 K 2023-11-28 03:55:26,867 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/d38dce7d1890ef61d80a87d7305594c5/.tmp/family2/152c70cd7cc74b44a1fdad7c99339763 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/d38dce7d1890ef61d80a87d7305594c5/family2/152c70cd7cc74b44a1fdad7c99339763 2023-11-28 03:55:26,868 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/8f7c6ece6778b5f8b473ef7daf18a3c4/.tmp/family2/3a331bbd4c8547858f2b52ac47ada923 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/8f7c6ece6778b5f8b473ef7daf18a3c4/family2/3a331bbd4c8547858f2b52ac47ada923 2023-11-28 03:55:26,871 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/d38dce7d1890ef61d80a87d7305594c5/family2/152c70cd7cc74b44a1fdad7c99339763, entries=2, sequenceid=13, filesize=5.1 K 2023-11-28 03:55:26,871 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/8f7c6ece6778b5f8b473ef7daf18a3c4/family2/3a331bbd4c8547858f2b52ac47ada923, entries=2, sequenceid=13, filesize=5.1 K 2023-11-28 03:55:26,872 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 8f7c6ece6778b5f8b473ef7daf18a3c4 in 884ms, sequenceid=13, compaction requested=true 2023-11-28 03:55:26,872 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for d38dce7d1890ef61d80a87d7305594c5 in 884ms, sequenceid=13, compaction requested=true 2023-11-28 03:55:26,872 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 8f7c6ece6778b5f8b473ef7daf18a3c4: 2023-11-28 03:55:26,872 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for d38dce7d1890ef61d80a87d7305594c5: 2023-11-28 03:55:26,872 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,2,1701143722033.d38dce7d1890ef61d80a87d7305594c5. 2023-11-28 03:55:26,872 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,,1701143722033.8f7c6ece6778b5f8b473ef7daf18a3c4. 2023-11-28 03:55:26,872 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-11-28 03:55:26,873 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-11-28 03:55:27,243 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/227f7071ca3e7b6612f5a149eeb75e98/.tmp/family2/8775d52f6774471e9646d07951b547c0 2023-11-28 03:55:27,247 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/227f7071ca3e7b6612f5a149eeb75e98/.tmp/family1/07fcc1ac132145d298a6f5bf416de5b0 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/227f7071ca3e7b6612f5a149eeb75e98/family1/07fcc1ac132145d298a6f5bf416de5b0 2023-11-28 03:55:27,251 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/227f7071ca3e7b6612f5a149eeb75e98/family1/07fcc1ac132145d298a6f5bf416de5b0, entries=4, sequenceid=13, filesize=5.4 K 2023-11-28 03:55:27,251 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/227f7071ca3e7b6612f5a149eeb75e98/.tmp/family2/8775d52f6774471e9646d07951b547c0 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/227f7071ca3e7b6612f5a149eeb75e98/family2/8775d52f6774471e9646d07951b547c0 2023-11-28 03:55:27,255 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/227f7071ca3e7b6612f5a149eeb75e98/family2/8775d52f6774471e9646d07951b547c0, entries=2, sequenceid=13, filesize=5.1 K 2023-11-28 03:55:27,255 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 227f7071ca3e7b6612f5a149eeb75e98 in 1268ms, sequenceid=13, compaction requested=true 2023-11-28 03:55:27,255 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 227f7071ca3e7b6612f5a149eeb75e98: 2023-11-28 03:55:27,255 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,1,1701143722033.227f7071ca3e7b6612f5a149eeb75e98. 2023-11-28 03:55:27,256 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-11-28 03:55:27,256 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-11-28 03:55:27,256 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:27,256 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns:testMinorCompaction' locally acquired 2023-11-28 03:55:27,256 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure (ns:testMinorCompaction) in zk 2023-11-28 03:55:27,257 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:27,257 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-28 03:55:27,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:27,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:55:27,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:55:27,258 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-28 03:55:27,258 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns:testMinorCompaction' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-28 03:55:27,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:55:27,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:55:27,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-11-28 03:55:27,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:27,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:55:27,259 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure 'ns:testMinorCompaction' on coordinator 2023-11-28 03:55:27,259 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@17912b98[Count = 0] remaining members to acquire global barrier 2023-11-28 03:55:27,259 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns:testMinorCompaction' starting 'in-barrier' execution. 2023-11-28 03:55:27,259 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-28 03:55:27,260 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-28 03:55:27,260 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-28 03:55:27,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-28 03:55:27,261 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns:testMinorCompaction' received 'reached' from coordinator. 2023-11-28 03:55:27,261 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns:testMinorCompaction' locally completed 2023-11-28 03:55:27,261 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns:testMinorCompaction' completed for member 'jenkins-hbase4.apache.org,41203,1701143600653' in zk 2023-11-28 03:55:27,261 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:27,261 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-28 03:55:27,268 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns:testMinorCompaction' has notified controller of completion 2023-11-28 03:55:27,268 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:27,268 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:55:27,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:27,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:55:27,268 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns:testMinorCompaction' completed. 2023-11-28 03:55:27,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:55:27,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:55:27,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:55:27,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-11-28 03:55:27,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:27,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:55:27,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-11-28 03:55:27,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:27,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'ns:testMinorCompaction' member 'jenkins-hbase4.apache.org,41203,1701143600653': 2023-11-28 03:55:27,271 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' released barrier for procedure'ns:testMinorCompaction', counting down latch. Waiting for 0 more 2023-11-28 03:55:27,271 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns:testMinorCompaction' execution completed 2023-11-28 03:55:27,271 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-28 03:55:27,271 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-28 03:55:27,271 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns:testMinorCompaction 2023-11-28 03:55:27,271 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns:testMinorCompactionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-28 03:55:27,272 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-28 03:55:27,272 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-28 03:55:27,272 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-28 03:55:27,272 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:55:27,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-28 03:55:27,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-28 03:55:27,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:55:27,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:55:27,273 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:55:27,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:55:27,273 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:27,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:55:27,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-28 03:55:27,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-11-28 03:55:27,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:55:27,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-11-28 03:55:27,274 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:27,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:27,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:55:27,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-11-28 03:55:27,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:27,281 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:27,281 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:55:27,281 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-11-28 03:55:27,281 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:55:27,281 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns:testMinorCompaction' 2023-11-28 03:55:27,281 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-28 03:55:27,281 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:55:27,281 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-11-28 03:55:27,281 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:55:27,281 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:27,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:55:27,281 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-28 03:55:27,281 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns:testMinorCompaction'' to complete. (max 300000 ms per retry) 2023-11-28 03:55:27,281 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-28 03:55:27,281 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-28 03:55:27,281 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-28 03:55:27,281 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:55:27,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:55:27,381 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-28 03:55:27,382 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-28 03:55:27,383 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41203] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,,1701143722033.8f7c6ece6778b5f8b473ef7daf18a3c4. with WAL disabled. Data may be lost in the event of a crash. 2023-11-28 03:55:27,384 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41203] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,1,1701143722033.227f7071ca3e7b6612f5a149eeb75e98. with WAL disabled. Data may be lost in the event of a crash. 2023-11-28 03:55:27,384 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41203] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,2,1701143722033.d38dce7d1890ef61d80a87d7305594c5. with WAL disabled. Data may be lost in the event of a crash. 2023-11-28 03:55:27,386 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-28 03:55:27,387 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] procedure.ProcedureCoordinator(143): Procedure ns:testMinorCompaction was in running list but was completed. Accepting new attempt. 2023-11-28 03:55:27,388 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] procedure.ProcedureCoordinator(165): Submitting procedure ns:testMinorCompaction 2023-11-28 03:55:27,388 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns:testMinorCompaction' 2023-11-28 03:55:27,388 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:55:27,388 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns:testMinorCompaction' starting 'acquire' 2023-11-28 03:55:27,388 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns:testMinorCompaction', kicking off acquire phase on members. 2023-11-28 03:55:27,388 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-28 03:55:27,388 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-11-28 03:55:27,390 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:55:27,390 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:27,390 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:55:27,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:55:27,390 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:27,390 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-28 03:55:27,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-11-28 03:55:27,391 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-28 03:55:27,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-28 03:55:27,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-11-28 03:55:27,391 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns:testMinorCompaction 2023-11-28 03:55:27,391 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc ns:testMinorCompaction is still present, removing 2023-11-28 03:55:27,391 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns:testMinorCompaction 2023-11-28 03:55:27,391 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns:testMinorCompaction' with timeout 60000ms 2023-11-28 03:55:27,391 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:55:27,391 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns:testMinorCompaction' starting 'acquire' stage 2023-11-28 03:55:27,392 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,1,1701143722033.227f7071ca3e7b6612f5a149eeb75e98. 2023-11-28 03:55:27,392 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,1,1701143722033.227f7071ca3e7b6612f5a149eeb75e98. started... 2023-11-28 03:55:27,392 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-11-28 03:55:27,392 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-28 03:55:27,392 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,,1701143722033.8f7c6ece6778b5f8b473ef7daf18a3c4. 2023-11-28 03:55:27,392 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 227f7071ca3e7b6612f5a149eeb75e98 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-11-28 03:55:27,392 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,,1701143722033.8f7c6ece6778b5f8b473ef7daf18a3c4. started... 2023-11-28 03:55:27,392 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,2,1701143722033.d38dce7d1890ef61d80a87d7305594c5. 2023-11-28 03:55:27,392 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 8f7c6ece6778b5f8b473ef7daf18a3c4 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-11-28 03:55:27,392 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,2,1701143722033.d38dce7d1890ef61d80a87d7305594c5. started... 2023-11-28 03:55:27,392 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing d38dce7d1890ef61d80a87d7305594c5 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-11-28 03:55:27,412 INFO [Finalizer] client.ConnectionImplementation(1973): Closing master protocol: MasterService 2023-11-28 03:55:27,413 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x388cfe85 to 127.0.0.1:55641 2023-11-28 03:55:27,413 DEBUG [Finalizer] ipc.AbstractRpcClient(489): Stopping rpc client 2023-11-28 03:55:27,413 INFO [Finalizer] client.ConnectionImplementation(1973): Closing master protocol: MasterService 2023-11-28 03:55:27,413 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x447597be to 127.0.0.1:55641 2023-11-28 03:55:27,413 DEBUG [Finalizer] ipc.AbstractRpcClient(489): Stopping rpc client 2023-11-28 03:55:27,426 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20231128664fb1ee17b54797948f7e18fb012697 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e20231128664fb1ee17b54797948f7e18fb012697 2023-11-28 03:55:27,426 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023112838d457288a324d6face8101751b1e28d to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b2023112838d457288a324d6face8101751b1e28d 2023-11-28 03:55:27,427 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202311284d70390beba0415988c72c7d726c1d97 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c202311284d70390beba0415988c72c7d726c1d97 2023-11-28 03:55:27,431 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/d38dce7d1890ef61d80a87d7305594c5/.tmp/family1/0540db731c0d45d9b16a09a577bb8195 2023-11-28 03:55:27,831 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/8f7c6ece6778b5f8b473ef7daf18a3c4/.tmp/family1/2c2fd98a75144b91837125eddedc6b40 2023-11-28 03:55:27,831 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/227f7071ca3e7b6612f5a149eeb75e98/.tmp/family1/c0687dba89c24b8eadb93d3cc31f408d 2023-11-28 03:55:27,843 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202311281565b16740b94a2ea3087169007efa8d to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c81e728d9d4c2f636f067f89cc14862c202311281565b16740b94a2ea3087169007efa8d 2023-11-28 03:55:27,845 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20231128ac74c123299b45eb92df6065756f4310 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c4ca4238a0b923820dcc509a6f75849b20231128ac74c123299b45eb92df6065756f4310 2023-11-28 03:55:27,848 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/d38dce7d1890ef61d80a87d7305594c5/.tmp/family2/c7c382d7792f44ff987f3f70f5b05f25 2023-11-28 03:55:27,852 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/d38dce7d1890ef61d80a87d7305594c5/.tmp/family1/0540db731c0d45d9b16a09a577bb8195 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/d38dce7d1890ef61d80a87d7305594c5/family1/0540db731c0d45d9b16a09a577bb8195 2023-11-28 03:55:27,856 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/d38dce7d1890ef61d80a87d7305594c5/family1/0540db731c0d45d9b16a09a577bb8195, entries=4, sequenceid=17, filesize=5.4 K 2023-11-28 03:55:27,856 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/d38dce7d1890ef61d80a87d7305594c5/.tmp/family2/c7c382d7792f44ff987f3f70f5b05f25 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/d38dce7d1890ef61d80a87d7305594c5/family2/c7c382d7792f44ff987f3f70f5b05f25 2023-11-28 03:55:27,860 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/d38dce7d1890ef61d80a87d7305594c5/family2/c7c382d7792f44ff987f3f70f5b05f25, entries=2, sequenceid=17, filesize=5.1 K 2023-11-28 03:55:27,861 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for d38dce7d1890ef61d80a87d7305594c5 in 469ms, sequenceid=17, compaction requested=true 2023-11-28 03:55:27,861 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for d38dce7d1890ef61d80a87d7305594c5: 2023-11-28 03:55:27,861 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,2,1701143722033.d38dce7d1890ef61d80a87d7305594c5. 2023-11-28 03:55:27,861 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-11-28 03:55:28,245 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202311283a9de4b759d6465ca371ae673fe53883 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/d41d8cd98f00b204e9800998ecf8427e202311283a9de4b759d6465ca371ae673fe53883 2023-11-28 03:55:28,252 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/227f7071ca3e7b6612f5a149eeb75e98/.tmp/family2/9a4e79ab4bd9430d937fb52c4ad6b015 2023-11-28 03:55:28,255 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/227f7071ca3e7b6612f5a149eeb75e98/.tmp/family1/c0687dba89c24b8eadb93d3cc31f408d as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/227f7071ca3e7b6612f5a149eeb75e98/family1/c0687dba89c24b8eadb93d3cc31f408d 2023-11-28 03:55:28,258 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/227f7071ca3e7b6612f5a149eeb75e98/family1/c0687dba89c24b8eadb93d3cc31f408d, entries=4, sequenceid=17, filesize=5.4 K 2023-11-28 03:55:28,259 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/227f7071ca3e7b6612f5a149eeb75e98/.tmp/family2/9a4e79ab4bd9430d937fb52c4ad6b015 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/227f7071ca3e7b6612f5a149eeb75e98/family2/9a4e79ab4bd9430d937fb52c4ad6b015 2023-11-28 03:55:28,262 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/227f7071ca3e7b6612f5a149eeb75e98/family2/9a4e79ab4bd9430d937fb52c4ad6b015, entries=2, sequenceid=17, filesize=5.1 K 2023-11-28 03:55:28,263 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 227f7071ca3e7b6612f5a149eeb75e98 in 871ms, sequenceid=17, compaction requested=true 2023-11-28 03:55:28,263 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 227f7071ca3e7b6612f5a149eeb75e98: 2023-11-28 03:55:28,263 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,1,1701143722033.227f7071ca3e7b6612f5a149eeb75e98. 2023-11-28 03:55:28,263 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-11-28 03:55:28,340 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-11-28 03:55:28,649 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/8f7c6ece6778b5f8b473ef7daf18a3c4/.tmp/family2/75f85efbd7664901ba75b5cc5b6f4f38 2023-11-28 03:55:28,654 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/8f7c6ece6778b5f8b473ef7daf18a3c4/.tmp/family1/2c2fd98a75144b91837125eddedc6b40 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/8f7c6ece6778b5f8b473ef7daf18a3c4/family1/2c2fd98a75144b91837125eddedc6b40 2023-11-28 03:55:28,657 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/8f7c6ece6778b5f8b473ef7daf18a3c4/family1/2c2fd98a75144b91837125eddedc6b40, entries=4, sequenceid=17, filesize=5.4 K 2023-11-28 03:55:28,658 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/8f7c6ece6778b5f8b473ef7daf18a3c4/.tmp/family2/75f85efbd7664901ba75b5cc5b6f4f38 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/8f7c6ece6778b5f8b473ef7daf18a3c4/family2/75f85efbd7664901ba75b5cc5b6f4f38 2023-11-28 03:55:28,661 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/8f7c6ece6778b5f8b473ef7daf18a3c4/family2/75f85efbd7664901ba75b5cc5b6f4f38, entries=2, sequenceid=17, filesize=5.1 K 2023-11-28 03:55:28,662 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 8f7c6ece6778b5f8b473ef7daf18a3c4 in 1270ms, sequenceid=17, compaction requested=true 2023-11-28 03:55:28,662 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 8f7c6ece6778b5f8b473ef7daf18a3c4: 2023-11-28 03:55:28,662 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,,1701143722033.8f7c6ece6778b5f8b473ef7daf18a3c4. 2023-11-28 03:55:28,662 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-11-28 03:55:28,662 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-11-28 03:55:28,662 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:28,662 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns:testMinorCompaction' locally acquired 2023-11-28 03:55:28,662 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure (ns:testMinorCompaction) in zk 2023-11-28 03:55:28,665 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-28 03:55:28,665 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:28,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:28,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:55:28,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:55:28,665 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-28 03:55:28,665 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns:testMinorCompaction' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-28 03:55:28,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:55:28,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:55:28,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-11-28 03:55:28,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:28,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:55:28,666 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure 'ns:testMinorCompaction' on coordinator 2023-11-28 03:55:28,666 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@183618fd[Count = 0] remaining members to acquire global barrier 2023-11-28 03:55:28,666 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns:testMinorCompaction' starting 'in-barrier' execution. 2023-11-28 03:55:28,666 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-28 03:55:28,668 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-28 03:55:28,668 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-28 03:55:28,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-28 03:55:28,668 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns:testMinorCompaction' received 'reached' from coordinator. 2023-11-28 03:55:28,668 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns:testMinorCompaction' locally completed 2023-11-28 03:55:28,668 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns:testMinorCompaction' completed for member 'jenkins-hbase4.apache.org,41203,1701143600653' in zk 2023-11-28 03:55:28,668 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:28,668 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-28 03:55:28,669 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns:testMinorCompaction' has notified controller of completion 2023-11-28 03:55:28,669 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:55:28,669 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:28,669 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns:testMinorCompaction' completed. 2023-11-28 03:55:28,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:28,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:55:28,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:55:28,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:55:28,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:55:28,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-11-28 03:55:28,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:28,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:55:28,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-11-28 03:55:28,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:28,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'ns:testMinorCompaction' member 'jenkins-hbase4.apache.org,41203,1701143600653': 2023-11-28 03:55:28,671 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' released barrier for procedure'ns:testMinorCompaction', counting down latch. Waiting for 0 more 2023-11-28 03:55:28,671 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns:testMinorCompaction' execution completed 2023-11-28 03:55:28,671 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-28 03:55:28,671 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-28 03:55:28,671 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns:testMinorCompaction 2023-11-28 03:55:28,671 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns:testMinorCompactionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-28 03:55:28,673 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-28 03:55:28,673 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-28 03:55:28,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-28 03:55:28,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:55:28,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:55:28,673 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-28 03:55:28,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-28 03:55:28,673 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:55:28,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:55:28,673 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:55:28,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:55:28,673 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:28,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-11-28 03:55:28,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-28 03:55:28,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:55:28,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-11-28 03:55:28,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:28,674 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:28,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:55:28,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-11-28 03:55:28,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:28,677 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:28,677 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:55:28,677 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-11-28 03:55:28,678 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns:testMinorCompaction' 2023-11-28 03:55:28,678 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-28 03:55:28,678 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-11-28 03:55:28,678 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:55:28,678 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:55:28,678 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:55:28,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:55:28,678 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns:testMinorCompaction'' to complete. (max 300000 ms per retry) 2023-11-28 03:55:28,678 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-28 03:55:28,678 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:28,678 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-28 03:55:28,678 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-28 03:55:28,678 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-28 03:55:28,678 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:55:28,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:55:28,778 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-28 03:55:28,779 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-28 03:55:28,790 DEBUG [Listener at localhost/42853] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-11-28 03:55:28,799 DEBUG [Listener at localhost/42853] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-11-28 03:55:28,807 DEBUG [Listener at localhost/42853] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-11-28 03:55:28,808 DEBUG [Listener at localhost/42853] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-11-28 03:55:28,833 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-28 03:55:28,834 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] procedure.ProcedureCoordinator(143): Procedure ns:testMinorCompaction was in running list but was completed. Accepting new attempt. 2023-11-28 03:55:28,834 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] procedure.ProcedureCoordinator(165): Submitting procedure ns:testMinorCompaction 2023-11-28 03:55:28,834 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns:testMinorCompaction' 2023-11-28 03:55:28,835 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:55:28,835 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns:testMinorCompaction' starting 'acquire' 2023-11-28 03:55:28,835 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns:testMinorCompaction', kicking off acquire phase on members. 2023-11-28 03:55:28,835 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-28 03:55:28,835 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-11-28 03:55:28,836 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:55:28,836 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:28,836 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:55:28,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:55:28,837 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:28,837 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-28 03:55:28,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-11-28 03:55:28,837 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-28 03:55:28,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-28 03:55:28,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-11-28 03:55:28,837 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns:testMinorCompaction 2023-11-28 03:55:28,838 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc ns:testMinorCompaction is still present, removing 2023-11-28 03:55:28,838 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns:testMinorCompaction 2023-11-28 03:55:28,838 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns:testMinorCompaction' with timeout 60000ms 2023-11-28 03:55:28,838 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:55:28,838 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns:testMinorCompaction' starting 'acquire' stage 2023-11-28 03:55:28,838 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,1,1701143722033.227f7071ca3e7b6612f5a149eeb75e98. 2023-11-28 03:55:28,838 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-11-28 03:55:28,838 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-28 03:55:28,838 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,1,1701143722033.227f7071ca3e7b6612f5a149eeb75e98. started... 2023-11-28 03:55:28,838 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,2,1701143722033.d38dce7d1890ef61d80a87d7305594c5. 2023-11-28 03:55:28,838 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,,1701143722033.8f7c6ece6778b5f8b473ef7daf18a3c4. 2023-11-28 03:55:28,838 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 227f7071ca3e7b6612f5a149eeb75e98 2/2 column families, dataSize=138 B heapSize=944 B 2023-11-28 03:55:28,838 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,2,1701143722033.d38dce7d1890ef61d80a87d7305594c5. started... 2023-11-28 03:55:28,838 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,,1701143722033.8f7c6ece6778b5f8b473ef7daf18a3c4. started... 2023-11-28 03:55:28,838 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing d38dce7d1890ef61d80a87d7305594c5 2/2 column families, dataSize=138 B heapSize=944 B 2023-11-28 03:55:28,839 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 8f7c6ece6778b5f8b473ef7daf18a3c4 2/2 column families, dataSize=138 B heapSize=944 B 2023-11-28 03:55:28,864 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/8f7c6ece6778b5f8b473ef7daf18a3c4/.tmp/family1/2b76626a5c574a1ca2e46dc4a508711f 2023-11-28 03:55:28,867 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2b76626a5c574a1ca2e46dc4a508711f 2023-11-28 03:55:29,279 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/8f7c6ece6778b5f8b473ef7daf18a3c4/.tmp/family2/2cdd2e95af8f44d4b5a8f6e5485c91b5 2023-11-28 03:55:29,282 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2cdd2e95af8f44d4b5a8f6e5485c91b5 2023-11-28 03:55:29,283 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/8f7c6ece6778b5f8b473ef7daf18a3c4/.tmp/family1/2b76626a5c574a1ca2e46dc4a508711f as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/8f7c6ece6778b5f8b473ef7daf18a3c4/family1/2b76626a5c574a1ca2e46dc4a508711f 2023-11-28 03:55:29,286 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2b76626a5c574a1ca2e46dc4a508711f 2023-11-28 03:55:29,286 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/8f7c6ece6778b5f8b473ef7daf18a3c4/family1/2b76626a5c574a1ca2e46dc4a508711f, entries=3, sequenceid=23, filesize=5.0 K 2023-11-28 03:55:29,287 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/8f7c6ece6778b5f8b473ef7daf18a3c4/.tmp/family2/2cdd2e95af8f44d4b5a8f6e5485c91b5 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/8f7c6ece6778b5f8b473ef7daf18a3c4/family2/2cdd2e95af8f44d4b5a8f6e5485c91b5 2023-11-28 03:55:29,290 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2cdd2e95af8f44d4b5a8f6e5485c91b5 2023-11-28 03:55:29,290 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/8f7c6ece6778b5f8b473ef7daf18a3c4/family2/2cdd2e95af8f44d4b5a8f6e5485c91b5, entries=1, sequenceid=23, filesize=5.0 K 2023-11-28 03:55:29,291 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 8f7c6ece6778b5f8b473ef7daf18a3c4 in 452ms, sequenceid=23, compaction requested=true 2023-11-28 03:55:29,291 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 8f7c6ece6778b5f8b473ef7daf18a3c4: 2023-11-28 03:55:29,291 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,,1701143722033.8f7c6ece6778b5f8b473ef7daf18a3c4. 2023-11-28 03:55:29,291 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-11-28 03:55:29,656 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/d38dce7d1890ef61d80a87d7305594c5/.tmp/family1/32a25ebf8f0541bf82b633333515fb26 2023-11-28 03:55:29,657 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/227f7071ca3e7b6612f5a149eeb75e98/.tmp/family1/af0d088c6b8d4b15ad62b105aefb4493 2023-11-28 03:55:29,660 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 32a25ebf8f0541bf82b633333515fb26 2023-11-28 03:55:29,661 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for af0d088c6b8d4b15ad62b105aefb4493 2023-11-28 03:55:30,084 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/d38dce7d1890ef61d80a87d7305594c5/.tmp/family2/6631602ed54d4496b3d1c556c861c276 2023-11-28 03:55:30,084 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/227f7071ca3e7b6612f5a149eeb75e98/.tmp/family2/d782cde5e6a84c6aae3a87c2dc9af55d 2023-11-28 03:55:30,088 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6631602ed54d4496b3d1c556c861c276 2023-11-28 03:55:30,088 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d782cde5e6a84c6aae3a87c2dc9af55d 2023-11-28 03:55:30,089 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/d38dce7d1890ef61d80a87d7305594c5/.tmp/family1/32a25ebf8f0541bf82b633333515fb26 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/d38dce7d1890ef61d80a87d7305594c5/family1/32a25ebf8f0541bf82b633333515fb26 2023-11-28 03:55:30,089 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/227f7071ca3e7b6612f5a149eeb75e98/.tmp/family1/af0d088c6b8d4b15ad62b105aefb4493 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/227f7071ca3e7b6612f5a149eeb75e98/family1/af0d088c6b8d4b15ad62b105aefb4493 2023-11-28 03:55:30,093 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 32a25ebf8f0541bf82b633333515fb26 2023-11-28 03:55:30,093 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/d38dce7d1890ef61d80a87d7305594c5/family1/32a25ebf8f0541bf82b633333515fb26, entries=3, sequenceid=23, filesize=5.0 K 2023-11-28 03:55:30,094 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for af0d088c6b8d4b15ad62b105aefb4493 2023-11-28 03:55:30,094 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/227f7071ca3e7b6612f5a149eeb75e98/family1/af0d088c6b8d4b15ad62b105aefb4493, entries=3, sequenceid=23, filesize=5.0 K 2023-11-28 03:55:30,094 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/d38dce7d1890ef61d80a87d7305594c5/.tmp/family2/6631602ed54d4496b3d1c556c861c276 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/d38dce7d1890ef61d80a87d7305594c5/family2/6631602ed54d4496b3d1c556c861c276 2023-11-28 03:55:30,095 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/227f7071ca3e7b6612f5a149eeb75e98/.tmp/family2/d782cde5e6a84c6aae3a87c2dc9af55d as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/227f7071ca3e7b6612f5a149eeb75e98/family2/d782cde5e6a84c6aae3a87c2dc9af55d 2023-11-28 03:55:30,099 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d782cde5e6a84c6aae3a87c2dc9af55d 2023-11-28 03:55:30,100 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/227f7071ca3e7b6612f5a149eeb75e98/family2/d782cde5e6a84c6aae3a87c2dc9af55d, entries=1, sequenceid=23, filesize=5.0 K 2023-11-28 03:55:30,100 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6631602ed54d4496b3d1c556c861c276 2023-11-28 03:55:30,100 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/d38dce7d1890ef61d80a87d7305594c5/family2/6631602ed54d4496b3d1c556c861c276, entries=1, sequenceid=23, filesize=5.0 K 2023-11-28 03:55:30,100 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 227f7071ca3e7b6612f5a149eeb75e98 in 1262ms, sequenceid=23, compaction requested=true 2023-11-28 03:55:30,100 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for d38dce7d1890ef61d80a87d7305594c5 in 1262ms, sequenceid=23, compaction requested=true 2023-11-28 03:55:30,100 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 227f7071ca3e7b6612f5a149eeb75e98: 2023-11-28 03:55:30,101 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for d38dce7d1890ef61d80a87d7305594c5: 2023-11-28 03:55:30,101 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,1,1701143722033.227f7071ca3e7b6612f5a149eeb75e98. 2023-11-28 03:55:30,101 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,2,1701143722033.d38dce7d1890ef61d80a87d7305594c5. 2023-11-28 03:55:30,101 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-11-28 03:55:30,101 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-11-28 03:55:30,101 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-11-28 03:55:30,101 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:30,101 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns:testMinorCompaction' locally acquired 2023-11-28 03:55:30,101 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure (ns:testMinorCompaction) in zk 2023-11-28 03:55:30,103 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:30,103 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-28 03:55:30,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:30,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:55:30,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:55:30,103 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-28 03:55:30,103 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns:testMinorCompaction' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-28 03:55:30,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:55:30,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:55:30,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-11-28 03:55:30,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:30,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:55:30,104 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure 'ns:testMinorCompaction' on coordinator 2023-11-28 03:55:30,104 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4ef121ee[Count = 0] remaining members to acquire global barrier 2023-11-28 03:55:30,104 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns:testMinorCompaction' starting 'in-barrier' execution. 2023-11-28 03:55:30,105 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-28 03:55:30,106 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-28 03:55:30,106 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-28 03:55:30,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-28 03:55:30,106 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns:testMinorCompaction' received 'reached' from coordinator. 2023-11-28 03:55:30,106 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns:testMinorCompaction' locally completed 2023-11-28 03:55:30,106 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:30,106 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-28 03:55:30,106 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns:testMinorCompaction' completed for member 'jenkins-hbase4.apache.org,41203,1701143600653' in zk 2023-11-28 03:55:30,108 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:30,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:30,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:55:30,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:55:30,109 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns:testMinorCompaction' has notified controller of completion 2023-11-28 03:55:30,109 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:55:30,109 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns:testMinorCompaction' completed. 2023-11-28 03:55:30,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:55:30,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:55:30,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-11-28 03:55:30,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:30,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:55:30,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-11-28 03:55:30,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:30,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'ns:testMinorCompaction' member 'jenkins-hbase4.apache.org,41203,1701143600653': 2023-11-28 03:55:30,111 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' released barrier for procedure'ns:testMinorCompaction', counting down latch. Waiting for 0 more 2023-11-28 03:55:30,111 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns:testMinorCompaction' execution completed 2023-11-28 03:55:30,112 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-28 03:55:30,112 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-28 03:55:30,112 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns:testMinorCompaction 2023-11-28 03:55:30,112 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns:testMinorCompactionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-28 03:55:30,113 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-28 03:55:30,114 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-28 03:55:30,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-28 03:55:30,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:55:30,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:55:30,114 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:55:30,114 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-28 03:55:30,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-28 03:55:30,114 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:30,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:55:30,114 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:55:30,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:55:30,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-11-28 03:55:30,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-28 03:55:30,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:55:30,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-11-28 03:55:30,115 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:30,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:30,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:55:30,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-11-28 03:55:30,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:30,118 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:30,118 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-11-28 03:55:30,118 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-11-28 03:55:30,118 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:55:30,118 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:30,118 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns:testMinorCompaction' 2023-11-28 03:55:30,118 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-28 03:55:30,118 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:55:30,118 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:55:30,118 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:55:30,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:55:30,118 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-28 03:55:30,118 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns:testMinorCompaction'' to complete. (max 300000 ms per retry) 2023-11-28 03:55:30,118 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(399): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/ns:testMinorCompaction because node does not exist (not an error) 2023-11-28 03:55:30,118 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-28 03:55:30,118 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-28 03:55:30,119 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:55:30,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:55:30,119 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-28 03:55:30,219 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-28 03:55:30,219 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-28 03:55:30,220 DEBUG [Listener at localhost/42853] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-11-28 03:55:30,220 DEBUG [Listener at localhost/42853] regionserver.HStore(1912): 227f7071ca3e7b6612f5a149eeb75e98/family1 is initiating major compaction (all files) 2023-11-28 03:55:30,220 INFO [Listener at localhost/42853] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-11-28 03:55:30,220 INFO [Listener at localhost/42853] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-11-28 03:55:30,220 INFO [Listener at localhost/42853] regionserver.HRegion(2259): Starting compaction of 227f7071ca3e7b6612f5a149eeb75e98/family1 in ns:testMinorCompaction,1,1701143722033.227f7071ca3e7b6612f5a149eeb75e98. 2023-11-28 03:55:30,220 INFO [Listener at localhost/42853] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/227f7071ca3e7b6612f5a149eeb75e98/family1/67113a419a8e409eb5707edeebe4c7e9, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/227f7071ca3e7b6612f5a149eeb75e98/family1/47d1fcd8025c4bd681fca058ee3dbec4, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/227f7071ca3e7b6612f5a149eeb75e98/family1/07fcc1ac132145d298a6f5bf416de5b0, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/227f7071ca3e7b6612f5a149eeb75e98/family1/c0687dba89c24b8eadb93d3cc31f408d, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/227f7071ca3e7b6612f5a149eeb75e98/family1/af0d088c6b8d4b15ad62b105aefb4493] into tmpdir=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/227f7071ca3e7b6612f5a149eeb75e98/.tmp, totalSize=26.6 K 2023-11-28 03:55:30,221 DEBUG [Listener at localhost/42853] compactions.Compactor(207): Compacting 67113a419a8e409eb5707edeebe4c7e9, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1701143723149 2023-11-28 03:55:30,221 DEBUG [Listener at localhost/42853] compactions.Compactor(207): Compacting 47d1fcd8025c4bd681fca058ee3dbec4, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1701143724969 2023-11-28 03:55:30,221 DEBUG [Listener at localhost/42853] compactions.Compactor(207): Compacting 07fcc1ac132145d298a6f5bf416de5b0, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1701143725978 2023-11-28 03:55:30,222 DEBUG [Listener at localhost/42853] compactions.Compactor(207): Compacting c0687dba89c24b8eadb93d3cc31f408d, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1701143727384 2023-11-28 03:55:30,222 DEBUG [Listener at localhost/42853] compactions.Compactor(207): Compacting af0d088c6b8d4b15ad62b105aefb4493, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-11-28 03:55:30,230 INFO [Listener at localhost/42853] throttle.PressureAwareThroughputController(145): 227f7071ca3e7b6612f5a149eeb75e98#family1#compaction#149 average throughput is 0.83 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-11-28 03:55:30,638 INFO [Listener at localhost/42853] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b2023112882a9f6e2146f474d833b065e8d174397_del 2023-11-28 03:55:30,638 INFO [Listener at localhost/42853] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023112882a9f6e2146f474d833b065e8d174397_del to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b2023112882a9f6e2146f474d833b065e8d174397_del 2023-11-28 03:55:31,450 INFO [Listener at localhost/42853] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a654442f5f27462886f0663217b74f9f 2023-11-28 03:55:31,451 DEBUG [Listener at localhost/42853] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/227f7071ca3e7b6612f5a149eeb75e98/.tmp/family1/a654442f5f27462886f0663217b74f9f as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/227f7071ca3e7b6612f5a149eeb75e98/family1/a654442f5f27462886f0663217b74f9f 2023-11-28 03:55:31,454 INFO [Listener at localhost/42853] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a654442f5f27462886f0663217b74f9f 2023-11-28 03:55:31,455 INFO [Listener at localhost/42853] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 227f7071ca3e7b6612f5a149eeb75e98/family1 of 227f7071ca3e7b6612f5a149eeb75e98 into a654442f5f27462886f0663217b74f9f(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-11-28 03:55:31,455 DEBUG [Listener at localhost/42853] regionserver.HRegion(2289): Compaction status journal for 227f7071ca3e7b6612f5a149eeb75e98: 2023-11-28 03:55:31,456 DEBUG [Listener at localhost/42853] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-11-28 03:55:31,456 DEBUG [Listener at localhost/42853] regionserver.HStore(1912): 227f7071ca3e7b6612f5a149eeb75e98/family2 is initiating major compaction (all files) 2023-11-28 03:55:31,456 INFO [Listener at localhost/42853] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-11-28 03:55:31,456 INFO [Listener at localhost/42853] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-11-28 03:55:31,456 INFO [Listener at localhost/42853] regionserver.HRegion(2259): Starting compaction of 227f7071ca3e7b6612f5a149eeb75e98/family2 in ns:testMinorCompaction,1,1701143722033.227f7071ca3e7b6612f5a149eeb75e98. 2023-11-28 03:55:31,456 INFO [Listener at localhost/42853] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/227f7071ca3e7b6612f5a149eeb75e98/family2/fbaad07b4ade42ca9a0cb454b0a3170a, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/227f7071ca3e7b6612f5a149eeb75e98/family2/71d360d7554c4350a5974ded5d9257b0, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/227f7071ca3e7b6612f5a149eeb75e98/family2/8775d52f6774471e9646d07951b547c0, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/227f7071ca3e7b6612f5a149eeb75e98/family2/9a4e79ab4bd9430d937fb52c4ad6b015, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/227f7071ca3e7b6612f5a149eeb75e98/family2/d782cde5e6a84c6aae3a87c2dc9af55d] into tmpdir=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/227f7071ca3e7b6612f5a149eeb75e98/.tmp, totalSize=25.3 K 2023-11-28 03:55:31,456 DEBUG [Listener at localhost/42853] compactions.Compactor(207): Compacting fbaad07b4ade42ca9a0cb454b0a3170a, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1701143723149 2023-11-28 03:55:31,457 DEBUG [Listener at localhost/42853] compactions.Compactor(207): Compacting 71d360d7554c4350a5974ded5d9257b0, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1701143724969 2023-11-28 03:55:31,457 DEBUG [Listener at localhost/42853] compactions.Compactor(207): Compacting 8775d52f6774471e9646d07951b547c0, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1701143725978 2023-11-28 03:55:31,457 DEBUG [Listener at localhost/42853] compactions.Compactor(207): Compacting 9a4e79ab4bd9430d937fb52c4ad6b015, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1701143727384 2023-11-28 03:55:31,458 DEBUG [Listener at localhost/42853] compactions.Compactor(207): Compacting d782cde5e6a84c6aae3a87c2dc9af55d, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-11-28 03:55:31,466 INFO [Listener at localhost/42853] throttle.PressureAwareThroughputController(145): 227f7071ca3e7b6612f5a149eeb75e98#family2#compaction#150 average throughput is 0.51 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-11-28 03:55:31,509 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/227f7071ca3e7b6612f5a149eeb75e98/family1/67113a419a8e409eb5707edeebe4c7e9, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/227f7071ca3e7b6612f5a149eeb75e98/family1/47d1fcd8025c4bd681fca058ee3dbec4, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/227f7071ca3e7b6612f5a149eeb75e98/family1/07fcc1ac132145d298a6f5bf416de5b0, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/227f7071ca3e7b6612f5a149eeb75e98/family1/c0687dba89c24b8eadb93d3cc31f408d, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/227f7071ca3e7b6612f5a149eeb75e98/family1/af0d088c6b8d4b15ad62b105aefb4493] to archive 2023-11-28 03:55:31,510 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-28 03:55:31,511 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/227f7071ca3e7b6612f5a149eeb75e98/family1/67113a419a8e409eb5707edeebe4c7e9 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/ns/testMinorCompaction/227f7071ca3e7b6612f5a149eeb75e98/family1/67113a419a8e409eb5707edeebe4c7e9 2023-11-28 03:55:31,512 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/227f7071ca3e7b6612f5a149eeb75e98/family1/47d1fcd8025c4bd681fca058ee3dbec4 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/ns/testMinorCompaction/227f7071ca3e7b6612f5a149eeb75e98/family1/47d1fcd8025c4bd681fca058ee3dbec4 2023-11-28 03:55:31,513 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/227f7071ca3e7b6612f5a149eeb75e98/family1/07fcc1ac132145d298a6f5bf416de5b0 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/ns/testMinorCompaction/227f7071ca3e7b6612f5a149eeb75e98/family1/07fcc1ac132145d298a6f5bf416de5b0 2023-11-28 03:55:31,514 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/227f7071ca3e7b6612f5a149eeb75e98/family1/c0687dba89c24b8eadb93d3cc31f408d to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/ns/testMinorCompaction/227f7071ca3e7b6612f5a149eeb75e98/family1/c0687dba89c24b8eadb93d3cc31f408d 2023-11-28 03:55:31,515 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/227f7071ca3e7b6612f5a149eeb75e98/family1/af0d088c6b8d4b15ad62b105aefb4493 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/ns/testMinorCompaction/227f7071ca3e7b6612f5a149eeb75e98/family1/af0d088c6b8d4b15ad62b105aefb4493 2023-11-28 03:55:31,875 INFO [Listener at localhost/42853] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b20231128741651d07685415d9f3e4d89ac66cd7f_del 2023-11-28 03:55:31,875 INFO [Listener at localhost/42853] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20231128741651d07685415d9f3e4d89ac66cd7f_del to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c4ca4238a0b923820dcc509a6f75849b20231128741651d07685415d9f3e4d89ac66cd7f_del 2023-11-28 03:55:32,687 INFO [Listener at localhost/42853] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 52366ab7b2674e4aa58323c5e6d4ab4b 2023-11-28 03:55:32,688 DEBUG [Listener at localhost/42853] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/227f7071ca3e7b6612f5a149eeb75e98/.tmp/family2/52366ab7b2674e4aa58323c5e6d4ab4b as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/227f7071ca3e7b6612f5a149eeb75e98/family2/52366ab7b2674e4aa58323c5e6d4ab4b 2023-11-28 03:55:32,691 INFO [Listener at localhost/42853] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 52366ab7b2674e4aa58323c5e6d4ab4b 2023-11-28 03:55:32,692 INFO [Listener at localhost/42853] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 227f7071ca3e7b6612f5a149eeb75e98/family2 of 227f7071ca3e7b6612f5a149eeb75e98 into 52366ab7b2674e4aa58323c5e6d4ab4b(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-11-28 03:55:32,692 DEBUG [Listener at localhost/42853] regionserver.HRegion(2289): Compaction status journal for 227f7071ca3e7b6612f5a149eeb75e98: 2023-11-28 03:55:32,692 DEBUG [Listener at localhost/42853] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-11-28 03:55:32,692 DEBUG [Listener at localhost/42853] regionserver.HStore(1912): 8f7c6ece6778b5f8b473ef7daf18a3c4/family1 is initiating major compaction (all files) 2023-11-28 03:55:32,692 INFO [Listener at localhost/42853] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-11-28 03:55:32,692 INFO [Listener at localhost/42853] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-11-28 03:55:32,692 INFO [Listener at localhost/42853] regionserver.HRegion(2259): Starting compaction of 8f7c6ece6778b5f8b473ef7daf18a3c4/family1 in ns:testMinorCompaction,,1701143722033.8f7c6ece6778b5f8b473ef7daf18a3c4. 2023-11-28 03:55:32,692 INFO [Listener at localhost/42853] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/8f7c6ece6778b5f8b473ef7daf18a3c4/family1/f8b2846f7d5e4343b60b0ae9a79f9dba, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/8f7c6ece6778b5f8b473ef7daf18a3c4/family1/743e80d2d5f34d2b9a48382b09672de3, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/8f7c6ece6778b5f8b473ef7daf18a3c4/family1/2bc75edf0b2e4e57916ebe8f6947fc32, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/8f7c6ece6778b5f8b473ef7daf18a3c4/family1/2c2fd98a75144b91837125eddedc6b40, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/8f7c6ece6778b5f8b473ef7daf18a3c4/family1/2b76626a5c574a1ca2e46dc4a508711f] into tmpdir=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/8f7c6ece6778b5f8b473ef7daf18a3c4/.tmp, totalSize=26.6 K 2023-11-28 03:55:32,693 DEBUG [Listener at localhost/42853] compactions.Compactor(207): Compacting f8b2846f7d5e4343b60b0ae9a79f9dba, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1701143723148 2023-11-28 03:55:32,693 DEBUG [Listener at localhost/42853] compactions.Compactor(207): Compacting 743e80d2d5f34d2b9a48382b09672de3, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1701143724969 2023-11-28 03:55:32,693 DEBUG [Listener at localhost/42853] compactions.Compactor(207): Compacting 2bc75edf0b2e4e57916ebe8f6947fc32, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1701143725978 2023-11-28 03:55:32,693 DEBUG [Listener at localhost/42853] compactions.Compactor(207): Compacting 2c2fd98a75144b91837125eddedc6b40, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1701143727383 2023-11-28 03:55:32,694 DEBUG [Listener at localhost/42853] compactions.Compactor(207): Compacting 2b76626a5c574a1ca2e46dc4a508711f, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-11-28 03:55:32,700 INFO [Listener at localhost/42853] throttle.PressureAwareThroughputController(145): 8f7c6ece6778b5f8b473ef7daf18a3c4#family1#compaction#151 average throughput is 0.83 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-11-28 03:55:32,705 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/227f7071ca3e7b6612f5a149eeb75e98/family2/fbaad07b4ade42ca9a0cb454b0a3170a, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/227f7071ca3e7b6612f5a149eeb75e98/family2/71d360d7554c4350a5974ded5d9257b0, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/227f7071ca3e7b6612f5a149eeb75e98/family2/8775d52f6774471e9646d07951b547c0, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/227f7071ca3e7b6612f5a149eeb75e98/family2/9a4e79ab4bd9430d937fb52c4ad6b015, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/227f7071ca3e7b6612f5a149eeb75e98/family2/d782cde5e6a84c6aae3a87c2dc9af55d] to archive 2023-11-28 03:55:32,705 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-28 03:55:32,707 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/227f7071ca3e7b6612f5a149eeb75e98/family2/fbaad07b4ade42ca9a0cb454b0a3170a to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/ns/testMinorCompaction/227f7071ca3e7b6612f5a149eeb75e98/family2/fbaad07b4ade42ca9a0cb454b0a3170a 2023-11-28 03:55:32,708 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/227f7071ca3e7b6612f5a149eeb75e98/family2/71d360d7554c4350a5974ded5d9257b0 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/ns/testMinorCompaction/227f7071ca3e7b6612f5a149eeb75e98/family2/71d360d7554c4350a5974ded5d9257b0 2023-11-28 03:55:32,709 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/227f7071ca3e7b6612f5a149eeb75e98/family2/8775d52f6774471e9646d07951b547c0 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/ns/testMinorCompaction/227f7071ca3e7b6612f5a149eeb75e98/family2/8775d52f6774471e9646d07951b547c0 2023-11-28 03:55:32,710 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/227f7071ca3e7b6612f5a149eeb75e98/family2/9a4e79ab4bd9430d937fb52c4ad6b015 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/ns/testMinorCompaction/227f7071ca3e7b6612f5a149eeb75e98/family2/9a4e79ab4bd9430d937fb52c4ad6b015 2023-11-28 03:55:32,710 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/227f7071ca3e7b6612f5a149eeb75e98/family2/d782cde5e6a84c6aae3a87c2dc9af55d to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/ns/testMinorCompaction/227f7071ca3e7b6612f5a149eeb75e98/family2/d782cde5e6a84c6aae3a87c2dc9af55d 2023-11-28 03:55:33,108 INFO [Listener at localhost/42853] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e202311289f59dc156a284de790b7c4f04ff69972_del 2023-11-28 03:55:33,109 INFO [Listener at localhost/42853] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202311289f59dc156a284de790b7c4f04ff69972_del to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e202311289f59dc156a284de790b7c4f04ff69972_del 2023-11-28 03:55:33,521 INFO [Listener at localhost/42853] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0e6186ef0ced405091fc78b19254a71b 2023-11-28 03:55:33,522 DEBUG [Listener at localhost/42853] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/8f7c6ece6778b5f8b473ef7daf18a3c4/.tmp/family1/0e6186ef0ced405091fc78b19254a71b as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/8f7c6ece6778b5f8b473ef7daf18a3c4/family1/0e6186ef0ced405091fc78b19254a71b 2023-11-28 03:55:33,525 INFO [Listener at localhost/42853] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0e6186ef0ced405091fc78b19254a71b 2023-11-28 03:55:33,526 INFO [Listener at localhost/42853] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 8f7c6ece6778b5f8b473ef7daf18a3c4/family1 of 8f7c6ece6778b5f8b473ef7daf18a3c4 into 0e6186ef0ced405091fc78b19254a71b(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-11-28 03:55:33,526 DEBUG [Listener at localhost/42853] regionserver.HRegion(2289): Compaction status journal for 8f7c6ece6778b5f8b473ef7daf18a3c4: 2023-11-28 03:55:33,526 DEBUG [Listener at localhost/42853] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-11-28 03:55:33,526 DEBUG [Listener at localhost/42853] regionserver.HStore(1912): 8f7c6ece6778b5f8b473ef7daf18a3c4/family2 is initiating major compaction (all files) 2023-11-28 03:55:33,526 INFO [Listener at localhost/42853] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-11-28 03:55:33,526 INFO [Listener at localhost/42853] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-11-28 03:55:33,526 INFO [Listener at localhost/42853] regionserver.HRegion(2259): Starting compaction of 8f7c6ece6778b5f8b473ef7daf18a3c4/family2 in ns:testMinorCompaction,,1701143722033.8f7c6ece6778b5f8b473ef7daf18a3c4. 2023-11-28 03:55:33,526 INFO [Listener at localhost/42853] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/8f7c6ece6778b5f8b473ef7daf18a3c4/family2/068d7e9e4de34c42be56a2ee651a2882, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/8f7c6ece6778b5f8b473ef7daf18a3c4/family2/50b8f1072a5f40d4a3264d767aedb922, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/8f7c6ece6778b5f8b473ef7daf18a3c4/family2/3a331bbd4c8547858f2b52ac47ada923, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/8f7c6ece6778b5f8b473ef7daf18a3c4/family2/75f85efbd7664901ba75b5cc5b6f4f38, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/8f7c6ece6778b5f8b473ef7daf18a3c4/family2/2cdd2e95af8f44d4b5a8f6e5485c91b5] into tmpdir=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/8f7c6ece6778b5f8b473ef7daf18a3c4/.tmp, totalSize=25.3 K 2023-11-28 03:55:33,527 DEBUG [Listener at localhost/42853] compactions.Compactor(207): Compacting 068d7e9e4de34c42be56a2ee651a2882, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1701143723148 2023-11-28 03:55:33,527 DEBUG [Listener at localhost/42853] compactions.Compactor(207): Compacting 50b8f1072a5f40d4a3264d767aedb922, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1701143724969 2023-11-28 03:55:33,527 DEBUG [Listener at localhost/42853] compactions.Compactor(207): Compacting 3a331bbd4c8547858f2b52ac47ada923, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1701143725978 2023-11-28 03:55:33,527 DEBUG [Listener at localhost/42853] compactions.Compactor(207): Compacting 75f85efbd7664901ba75b5cc5b6f4f38, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1701143727383 2023-11-28 03:55:33,528 DEBUG [Listener at localhost/42853] compactions.Compactor(207): Compacting 2cdd2e95af8f44d4b5a8f6e5485c91b5, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-11-28 03:55:33,536 INFO [Listener at localhost/42853] throttle.PressureAwareThroughputController(145): 8f7c6ece6778b5f8b473ef7daf18a3c4#family2#compaction#152 average throughput is 0.51 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-11-28 03:55:33,605 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/8f7c6ece6778b5f8b473ef7daf18a3c4/family1/f8b2846f7d5e4343b60b0ae9a79f9dba, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/8f7c6ece6778b5f8b473ef7daf18a3c4/family1/743e80d2d5f34d2b9a48382b09672de3, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/8f7c6ece6778b5f8b473ef7daf18a3c4/family1/2bc75edf0b2e4e57916ebe8f6947fc32, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/8f7c6ece6778b5f8b473ef7daf18a3c4/family1/2c2fd98a75144b91837125eddedc6b40, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/8f7c6ece6778b5f8b473ef7daf18a3c4/family1/2b76626a5c574a1ca2e46dc4a508711f] to archive 2023-11-28 03:55:33,606 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-28 03:55:33,607 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/8f7c6ece6778b5f8b473ef7daf18a3c4/family1/f8b2846f7d5e4343b60b0ae9a79f9dba to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/ns/testMinorCompaction/8f7c6ece6778b5f8b473ef7daf18a3c4/family1/f8b2846f7d5e4343b60b0ae9a79f9dba 2023-11-28 03:55:33,608 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/8f7c6ece6778b5f8b473ef7daf18a3c4/family1/743e80d2d5f34d2b9a48382b09672de3 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/ns/testMinorCompaction/8f7c6ece6778b5f8b473ef7daf18a3c4/family1/743e80d2d5f34d2b9a48382b09672de3 2023-11-28 03:55:33,609 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/8f7c6ece6778b5f8b473ef7daf18a3c4/family1/2bc75edf0b2e4e57916ebe8f6947fc32 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/ns/testMinorCompaction/8f7c6ece6778b5f8b473ef7daf18a3c4/family1/2bc75edf0b2e4e57916ebe8f6947fc32 2023-11-28 03:55:33,610 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/8f7c6ece6778b5f8b473ef7daf18a3c4/family1/2c2fd98a75144b91837125eddedc6b40 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/ns/testMinorCompaction/8f7c6ece6778b5f8b473ef7daf18a3c4/family1/2c2fd98a75144b91837125eddedc6b40 2023-11-28 03:55:33,611 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/8f7c6ece6778b5f8b473ef7daf18a3c4/family1/2b76626a5c574a1ca2e46dc4a508711f to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/ns/testMinorCompaction/8f7c6ece6778b5f8b473ef7daf18a3c4/family1/2b76626a5c574a1ca2e46dc4a508711f 2023-11-28 03:55:33,943 INFO [Listener at localhost/42853] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e20231128ac96b96cb423452e9d3efb46dcf119e4_del 2023-11-28 03:55:33,943 INFO [Listener at localhost/42853] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20231128ac96b96cb423452e9d3efb46dcf119e4_del to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/d41d8cd98f00b204e9800998ecf8427e20231128ac96b96cb423452e9d3efb46dcf119e4_del 2023-11-28 03:55:34,758 INFO [Listener at localhost/42853] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 1f830d88dbb647e5b3158ad1e9866818 2023-11-28 03:55:34,759 DEBUG [Listener at localhost/42853] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/8f7c6ece6778b5f8b473ef7daf18a3c4/.tmp/family2/1f830d88dbb647e5b3158ad1e9866818 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/8f7c6ece6778b5f8b473ef7daf18a3c4/family2/1f830d88dbb647e5b3158ad1e9866818 2023-11-28 03:55:34,763 INFO [Listener at localhost/42853] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 1f830d88dbb647e5b3158ad1e9866818 2023-11-28 03:55:34,764 INFO [Listener at localhost/42853] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 8f7c6ece6778b5f8b473ef7daf18a3c4/family2 of 8f7c6ece6778b5f8b473ef7daf18a3c4 into 1f830d88dbb647e5b3158ad1e9866818(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-11-28 03:55:34,764 DEBUG [Listener at localhost/42853] regionserver.HRegion(2289): Compaction status journal for 8f7c6ece6778b5f8b473ef7daf18a3c4: 2023-11-28 03:55:34,764 DEBUG [Listener at localhost/42853] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-11-28 03:55:34,764 DEBUG [Listener at localhost/42853] regionserver.HStore(1912): d38dce7d1890ef61d80a87d7305594c5/family1 is initiating major compaction (all files) 2023-11-28 03:55:34,764 INFO [Listener at localhost/42853] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-11-28 03:55:34,764 INFO [Listener at localhost/42853] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-11-28 03:55:34,764 INFO [Listener at localhost/42853] regionserver.HRegion(2259): Starting compaction of d38dce7d1890ef61d80a87d7305594c5/family1 in ns:testMinorCompaction,2,1701143722033.d38dce7d1890ef61d80a87d7305594c5. 2023-11-28 03:55:34,765 INFO [Listener at localhost/42853] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/d38dce7d1890ef61d80a87d7305594c5/family1/0913518fbcf6490eac43fd7fdc96efab, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/d38dce7d1890ef61d80a87d7305594c5/family1/3ca3b28c49a54775b7a01e8c930bfd85, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/d38dce7d1890ef61d80a87d7305594c5/family1/32a8c14c2722489bb9a27951bdfa2146, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/d38dce7d1890ef61d80a87d7305594c5/family1/0540db731c0d45d9b16a09a577bb8195, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/d38dce7d1890ef61d80a87d7305594c5/family1/32a25ebf8f0541bf82b633333515fb26] into tmpdir=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/d38dce7d1890ef61d80a87d7305594c5/.tmp, totalSize=26.6 K 2023-11-28 03:55:34,765 DEBUG [Listener at localhost/42853] compactions.Compactor(207): Compacting 0913518fbcf6490eac43fd7fdc96efab, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1701143723149 2023-11-28 03:55:34,765 DEBUG [Listener at localhost/42853] compactions.Compactor(207): Compacting 3ca3b28c49a54775b7a01e8c930bfd85, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1701143724970 2023-11-28 03:55:34,766 DEBUG [Listener at localhost/42853] compactions.Compactor(207): Compacting 32a8c14c2722489bb9a27951bdfa2146, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1701143725978 2023-11-28 03:55:34,766 DEBUG [Listener at localhost/42853] compactions.Compactor(207): Compacting 0540db731c0d45d9b16a09a577bb8195, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1701143727384 2023-11-28 03:55:34,766 DEBUG [Listener at localhost/42853] compactions.Compactor(207): Compacting 32a25ebf8f0541bf82b633333515fb26, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-11-28 03:55:34,773 INFO [Listener at localhost/42853] throttle.PressureAwareThroughputController(145): d38dce7d1890ef61d80a87d7305594c5#family1#compaction#153 average throughput is 0.83 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-11-28 03:55:34,806 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/8f7c6ece6778b5f8b473ef7daf18a3c4/family2/068d7e9e4de34c42be56a2ee651a2882, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/8f7c6ece6778b5f8b473ef7daf18a3c4/family2/50b8f1072a5f40d4a3264d767aedb922, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/8f7c6ece6778b5f8b473ef7daf18a3c4/family2/3a331bbd4c8547858f2b52ac47ada923, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/8f7c6ece6778b5f8b473ef7daf18a3c4/family2/75f85efbd7664901ba75b5cc5b6f4f38, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/8f7c6ece6778b5f8b473ef7daf18a3c4/family2/2cdd2e95af8f44d4b5a8f6e5485c91b5] to archive 2023-11-28 03:55:34,806 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-28 03:55:34,808 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/8f7c6ece6778b5f8b473ef7daf18a3c4/family2/068d7e9e4de34c42be56a2ee651a2882 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/ns/testMinorCompaction/8f7c6ece6778b5f8b473ef7daf18a3c4/family2/068d7e9e4de34c42be56a2ee651a2882 2023-11-28 03:55:34,809 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/8f7c6ece6778b5f8b473ef7daf18a3c4/family2/50b8f1072a5f40d4a3264d767aedb922 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/ns/testMinorCompaction/8f7c6ece6778b5f8b473ef7daf18a3c4/family2/50b8f1072a5f40d4a3264d767aedb922 2023-11-28 03:55:34,810 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/8f7c6ece6778b5f8b473ef7daf18a3c4/family2/3a331bbd4c8547858f2b52ac47ada923 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/ns/testMinorCompaction/8f7c6ece6778b5f8b473ef7daf18a3c4/family2/3a331bbd4c8547858f2b52ac47ada923 2023-11-28 03:55:34,811 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/8f7c6ece6778b5f8b473ef7daf18a3c4/family2/75f85efbd7664901ba75b5cc5b6f4f38 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/ns/testMinorCompaction/8f7c6ece6778b5f8b473ef7daf18a3c4/family2/75f85efbd7664901ba75b5cc5b6f4f38 2023-11-28 03:55:34,812 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/8f7c6ece6778b5f8b473ef7daf18a3c4/family2/2cdd2e95af8f44d4b5a8f6e5485c91b5 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/ns/testMinorCompaction/8f7c6ece6778b5f8b473ef7daf18a3c4/family2/2cdd2e95af8f44d4b5a8f6e5485c91b5 2023-11-28 03:55:35,181 INFO [Listener at localhost/42853] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c202311281d0ae96816c443da9d1775d675376278_del 2023-11-28 03:55:35,181 INFO [Listener at localhost/42853] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202311281d0ae96816c443da9d1775d675376278_del to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c202311281d0ae96816c443da9d1775d675376278_del 2023-11-28 03:55:35,994 INFO [Listener at localhost/42853] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 87958e79536146f9917ab65455aa3da4 2023-11-28 03:55:35,994 DEBUG [Listener at localhost/42853] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/d38dce7d1890ef61d80a87d7305594c5/.tmp/family1/87958e79536146f9917ab65455aa3da4 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/d38dce7d1890ef61d80a87d7305594c5/family1/87958e79536146f9917ab65455aa3da4 2023-11-28 03:55:35,998 INFO [Listener at localhost/42853] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 87958e79536146f9917ab65455aa3da4 2023-11-28 03:55:35,999 INFO [Listener at localhost/42853] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in d38dce7d1890ef61d80a87d7305594c5/family1 of d38dce7d1890ef61d80a87d7305594c5 into 87958e79536146f9917ab65455aa3da4(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-11-28 03:55:35,999 DEBUG [Listener at localhost/42853] regionserver.HRegion(2289): Compaction status journal for d38dce7d1890ef61d80a87d7305594c5: 2023-11-28 03:55:35,999 DEBUG [Listener at localhost/42853] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-11-28 03:55:35,999 DEBUG [Listener at localhost/42853] regionserver.HStore(1912): d38dce7d1890ef61d80a87d7305594c5/family2 is initiating major compaction (all files) 2023-11-28 03:55:35,999 INFO [Listener at localhost/42853] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-11-28 03:55:35,999 INFO [Listener at localhost/42853] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-11-28 03:55:35,999 INFO [Listener at localhost/42853] regionserver.HRegion(2259): Starting compaction of d38dce7d1890ef61d80a87d7305594c5/family2 in ns:testMinorCompaction,2,1701143722033.d38dce7d1890ef61d80a87d7305594c5. 2023-11-28 03:55:35,999 INFO [Listener at localhost/42853] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/d38dce7d1890ef61d80a87d7305594c5/family2/844d4831eabb45148613e543e3187b99, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/d38dce7d1890ef61d80a87d7305594c5/family2/75dc829189a8422093cdb3bd57770ede, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/d38dce7d1890ef61d80a87d7305594c5/family2/152c70cd7cc74b44a1fdad7c99339763, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/d38dce7d1890ef61d80a87d7305594c5/family2/c7c382d7792f44ff987f3f70f5b05f25, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/d38dce7d1890ef61d80a87d7305594c5/family2/6631602ed54d4496b3d1c556c861c276] into tmpdir=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/d38dce7d1890ef61d80a87d7305594c5/.tmp, totalSize=25.3 K 2023-11-28 03:55:36,000 DEBUG [Listener at localhost/42853] compactions.Compactor(207): Compacting 844d4831eabb45148613e543e3187b99, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1701143723149 2023-11-28 03:55:36,000 DEBUG [Listener at localhost/42853] compactions.Compactor(207): Compacting 75dc829189a8422093cdb3bd57770ede, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1701143724970 2023-11-28 03:55:36,000 DEBUG [Listener at localhost/42853] compactions.Compactor(207): Compacting 152c70cd7cc74b44a1fdad7c99339763, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1701143725978 2023-11-28 03:55:36,001 DEBUG [Listener at localhost/42853] compactions.Compactor(207): Compacting c7c382d7792f44ff987f3f70f5b05f25, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1701143727384 2023-11-28 03:55:36,001 DEBUG [Listener at localhost/42853] compactions.Compactor(207): Compacting 6631602ed54d4496b3d1c556c861c276, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-11-28 03:55:36,005 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/d38dce7d1890ef61d80a87d7305594c5/family1/0913518fbcf6490eac43fd7fdc96efab, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/d38dce7d1890ef61d80a87d7305594c5/family1/3ca3b28c49a54775b7a01e8c930bfd85, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/d38dce7d1890ef61d80a87d7305594c5/family1/32a8c14c2722489bb9a27951bdfa2146, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/d38dce7d1890ef61d80a87d7305594c5/family1/0540db731c0d45d9b16a09a577bb8195, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/d38dce7d1890ef61d80a87d7305594c5/family1/32a25ebf8f0541bf82b633333515fb26] to archive 2023-11-28 03:55:36,006 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-28 03:55:36,008 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/d38dce7d1890ef61d80a87d7305594c5/family1/0913518fbcf6490eac43fd7fdc96efab to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/ns/testMinorCompaction/d38dce7d1890ef61d80a87d7305594c5/family1/0913518fbcf6490eac43fd7fdc96efab 2023-11-28 03:55:36,009 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/d38dce7d1890ef61d80a87d7305594c5/family1/3ca3b28c49a54775b7a01e8c930bfd85 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/ns/testMinorCompaction/d38dce7d1890ef61d80a87d7305594c5/family1/3ca3b28c49a54775b7a01e8c930bfd85 2023-11-28 03:55:36,010 INFO [Listener at localhost/42853] throttle.PressureAwareThroughputController(145): d38dce7d1890ef61d80a87d7305594c5#family2#compaction#154 average throughput is 0.34 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-11-28 03:55:36,010 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/d38dce7d1890ef61d80a87d7305594c5/family1/32a8c14c2722489bb9a27951bdfa2146 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/ns/testMinorCompaction/d38dce7d1890ef61d80a87d7305594c5/family1/32a8c14c2722489bb9a27951bdfa2146 2023-11-28 03:55:36,012 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/d38dce7d1890ef61d80a87d7305594c5/family1/0540db731c0d45d9b16a09a577bb8195 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/ns/testMinorCompaction/d38dce7d1890ef61d80a87d7305594c5/family1/0540db731c0d45d9b16a09a577bb8195 2023-11-28 03:55:36,017 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/d38dce7d1890ef61d80a87d7305594c5/family1/32a25ebf8f0541bf82b633333515fb26 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/ns/testMinorCompaction/d38dce7d1890ef61d80a87d7305594c5/family1/32a25ebf8f0541bf82b633333515fb26 2023-11-28 03:55:36,424 INFO [Listener at localhost/42853] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c20231128bdaf3475d2aa422ab143c4943d722215_del 2023-11-28 03:55:36,424 INFO [Listener at localhost/42853] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20231128bdaf3475d2aa422ab143c4943d722215_del to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c81e728d9d4c2f636f067f89cc14862c20231128bdaf3475d2aa422ab143c4943d722215_del 2023-11-28 03:55:37,239 INFO [Listener at localhost/42853] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c2867c59ef6249e083a7d534ad75de32 2023-11-28 03:55:37,240 DEBUG [Listener at localhost/42853] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/d38dce7d1890ef61d80a87d7305594c5/.tmp/family2/c2867c59ef6249e083a7d534ad75de32 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/d38dce7d1890ef61d80a87d7305594c5/family2/c2867c59ef6249e083a7d534ad75de32 2023-11-28 03:55:37,243 INFO [Listener at localhost/42853] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c2867c59ef6249e083a7d534ad75de32 2023-11-28 03:55:37,244 INFO [Listener at localhost/42853] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in d38dce7d1890ef61d80a87d7305594c5/family2 of d38dce7d1890ef61d80a87d7305594c5 into c2867c59ef6249e083a7d534ad75de32(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-11-28 03:55:37,244 DEBUG [Listener at localhost/42853] regionserver.HRegion(2289): Compaction status journal for d38dce7d1890ef61d80a87d7305594c5: 2023-11-28 03:55:37,247 DEBUG [Listener at localhost/42853] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-11-28 03:55:37,250 DEBUG [Listener at localhost/42853] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-11-28 03:55:37,253 DEBUG [Listener at localhost/42853] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-11-28 03:55:37,253 DEBUG [Listener at localhost/42853] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-11-28 03:55:37,257 INFO [Listener at localhost/42853] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-11-28 03:55:37,269 INFO [Listener at localhost/42853] compactions.PartitionedMobCompactor(306): The compaction type is PART_FILES, the request has 3 del files, 3 selected files, and 0 irrelevant files table 'ns:testMinorCompaction' and column 'family1' 2023-11-28 03:55:37,272 INFO [Listener at localhost/42853] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e202311289f59dc156a284de790b7c4f04ff69972_del 2023-11-28 03:55:37,274 INFO [Listener at localhost/42853] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b2023112882a9f6e2146f474d833b065e8d174397_del 2023-11-28 03:55:37,277 INFO [Listener at localhost/42853] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c202311281d0ae96816c443da9d1775d675376278_del 2023-11-28 03:55:37,277 INFO [Listener at localhost/42853] compactions.PartitionedMobCompactor(350): After merging, there are 3 del files. table='ns:testMinorCompaction' column='family1' 2023-11-28 03:55:37,278 DEBUG [Listener at localhost/42853] zookeeper.ReadOnlyZKClient(139): Connect 0x5abbae49 to 127.0.0.1:55641 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-11-28 03:55:37,282 DEBUG [Listener at localhost/42853] ipc.AbstractRpcClient(189): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@77715b, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-11-28 03:55:37,283 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20231128 for table='ns:testMinorCompaction' and column='family1' 2023-11-28 03:55:37,283 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c81e728d9d4c2f636f067f89cc14862c20231128 for table='ns:testMinorCompaction' and column='family1' 2023-11-28 03:55:37,283 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c4ca4238a0b923820dcc509a6f75849b20231128 for table='ns:testMinorCompaction' and column='family1' 2023-11-28 03:55:37,307 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/d38dce7d1890ef61d80a87d7305594c5/family2/844d4831eabb45148613e543e3187b99, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/d38dce7d1890ef61d80a87d7305594c5/family2/75dc829189a8422093cdb3bd57770ede, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/d38dce7d1890ef61d80a87d7305594c5/family2/152c70cd7cc74b44a1fdad7c99339763, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/d38dce7d1890ef61d80a87d7305594c5/family2/c7c382d7792f44ff987f3f70f5b05f25, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/d38dce7d1890ef61d80a87d7305594c5/family2/6631602ed54d4496b3d1c556c861c276] to archive 2023-11-28 03:55:37,308 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-28 03:55:37,310 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/d38dce7d1890ef61d80a87d7305594c5/family2/844d4831eabb45148613e543e3187b99 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/ns/testMinorCompaction/d38dce7d1890ef61d80a87d7305594c5/family2/844d4831eabb45148613e543e3187b99 2023-11-28 03:55:37,311 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/d38dce7d1890ef61d80a87d7305594c5/family2/75dc829189a8422093cdb3bd57770ede to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/ns/testMinorCompaction/d38dce7d1890ef61d80a87d7305594c5/family2/75dc829189a8422093cdb3bd57770ede 2023-11-28 03:55:37,313 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/d38dce7d1890ef61d80a87d7305594c5/family2/152c70cd7cc74b44a1fdad7c99339763 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/ns/testMinorCompaction/d38dce7d1890ef61d80a87d7305594c5/family2/152c70cd7cc74b44a1fdad7c99339763 2023-11-28 03:55:37,314 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/d38dce7d1890ef61d80a87d7305594c5/family2/c7c382d7792f44ff987f3f70f5b05f25 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/ns/testMinorCompaction/d38dce7d1890ef61d80a87d7305594c5/family2/c7c382d7792f44ff987f3f70f5b05f25 2023-11-28 03:55:37,315 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023112868e266e9da3d4ff08ae9e9fc0fb89c89 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e2023112868e266e9da3d4ff08ae9e9fc0fb89c89 2023-11-28 03:55:37,315 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/d38dce7d1890ef61d80a87d7305594c5/family2/6631602ed54d4496b3d1c556c861c276 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/ns/testMinorCompaction/d38dce7d1890ef61d80a87d7305594c5/family2/6631602ed54d4496b3d1c556c861c276 2023-11-28 03:55:37,315 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20231128 table='ns:testMinorCompaction' column='family1' 2023-11-28 03:55:37,328 DEBUG [MobFileCompactionChore-pool-0] ipc.RpcConnection(122): Using SIMPLE authentication for service=ClientService, sasl=false 2023-11-28 03:55:37,330 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:48330, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-11-28 03:55:37,335 DEBUG [MobFileCompactionChore-pool-0] ipc.RpcConnection(122): Using SIMPLE authentication for service=MasterService, sasl=false 2023-11-28 03:55:37,338 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:43942, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-11-28 03:55:37,350 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/ns/testMinorCompaction/d41d8cd98f00b204e9800998ecf8427e20231128/family1/228852b589c44374bd0c9222c2061f0d first=Optional[0\x00\x00\x00\x01] last=Optional[0\x00\x00\x00\x01] 2023-11-28 03:55:37,350 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=ns:testMinorCompaction,,1701143722033.8f7c6ece6778b5f8b473ef7daf18a3c4., hostname=jenkins-hbase4.apache.org,41203,1701143600653, seqNum=2 for row with hfile group [{family1,hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/ns/testMinorCompaction/d41d8cd98f00b204e9800998ecf8427e20231128/family1/228852b589c44374bd0c9222c2061f0d}] 2023-11-28 03:55:37,358 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41203] regionserver.HStore(763): Validating hfile at hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/ns/testMinorCompaction/d41d8cd98f00b204e9800998ecf8427e20231128/family1/228852b589c44374bd0c9222c2061f0d for inclusion in 8f7c6ece6778b5f8b473ef7daf18a3c4/family1 2023-11-28 03:55:37,361 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41203] regionserver.HStore(775): HFile bounds: first=0\x00\x00\x00\x01 last=0\x00\x00\x00\x01 2023-11-28 03:55:37,361 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41203] regionserver.HStore(777): Region bounds: first= last=1 2023-11-28 03:55:37,361 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41203] regionserver.HRegion(2446): Flush status journal for 8f7c6ece6778b5f8b473ef7daf18a3c4: 2023-11-28 03:55:37,362 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41203] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/ns/testMinorCompaction/d41d8cd98f00b204e9800998ecf8427e20231128/family1/228852b589c44374bd0c9222c2061f0d to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__ns_testMinorCompaction__bktnhvakfpktln3peav8d8gv9bvd86iop4c86i8rpu3lua7borbvo4j545am15u0/family1/228852b589c44374bd0c9222c2061f0d 2023-11-28 03:55:37,364 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41203] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__ns_testMinorCompaction__bktnhvakfpktln3peav8d8gv9bvd86iop4c86i8rpu3lua7borbvo4j545am15u0/family1/228852b589c44374bd0c9222c2061f0d as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/8f7c6ece6778b5f8b473ef7daf18a3c4/family1/2ae1c5cb057b47baa618949538125876_SeqId_28_ 2023-11-28 03:55:37,365 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41203] regionserver.HStore(849): Loaded HFile hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__ns_testMinorCompaction__bktnhvakfpktln3peav8d8gv9bvd86iop4c86i8rpu3lua7borbvo4j545am15u0/family1/228852b589c44374bd0c9222c2061f0d into 8f7c6ece6778b5f8b473ef7daf18a3c4/family1 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/8f7c6ece6778b5f8b473ef7daf18a3c4/family1/2ae1c5cb057b47baa618949538125876_SeqId_28_ - updating store file list. 2023-11-28 03:55:37,369 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41203] regionserver.HStore(882): Loaded HFile hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/8f7c6ece6778b5f8b473ef7daf18a3c4/family1/2ae1c5cb057b47baa618949538125876_SeqId_28_ into 8f7c6ece6778b5f8b473ef7daf18a3c4/family1 2023-11-28 03:55:37,369 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41203] regionserver.HStore(855): Successfully loaded hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__ns_testMinorCompaction__bktnhvakfpktln3peav8d8gv9bvd86iop4c86i8rpu3lua7borbvo4j545am15u0/family1/228852b589c44374bd0c9222c2061f0d into 8f7c6ece6778b5f8b473ef7daf18a3c4/family1 (new location: hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/8f7c6ece6778b5f8b473ef7daf18a3c4/family1/2ae1c5cb057b47baa618949538125876_SeqId_28_) 2023-11-28 03:55:37,370 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41203] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__ns_testMinorCompaction__bktnhvakfpktln3peav8d8gv9bvd86iop4c86i8rpu3lua7borbvo4j545am15u0/family1/228852b589c44374bd0c9222c2061f0d 2023-11-28 03:55:37,372 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20231128 table='ns:testMinorCompaction' column='family1' 2023-11-28 03:55:37,373 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-28 03:55:37,376 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e2023112865a3d186c99a4a8cbb01d7ce281c6a97 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e2023112865a3d186c99a4a8cbb01d7ce281c6a97 2023-11-28 03:55:37,376 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(508): Table ns:testMinorCompaction 1 expired mob files are deleted 2023-11-28 03:55:37,376 DEBUG [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/ns/testMinorCompaction/d41d8cd98f00b204e9800998ecf8427e20231128' 2023-11-28 03:55:37,377 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20231128 for table='ns:testMinorCompaction' and column='family1' 2023-11-28 03:55:38,116 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20231128707ba2b6a3a5493eac625003d2306c09 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b20231128707ba2b6a3a5493eac625003d2306c09 2023-11-28 03:55:38,116 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023112825add3e70607414c8ab0e21a558ebc96 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c2023112825add3e70607414c8ab0e21a558ebc96 2023-11-28 03:55:38,117 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20231128 table='ns:testMinorCompaction' column='family1' 2023-11-28 03:55:38,117 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20231128 table='ns:testMinorCompaction' column='family1' 2023-11-28 03:55:38,148 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c4ca4238a0b923820dcc509a6f75849b20231128/family1/6b09e18509fd4561a48a1a2828f2a4b3 first=Optional[1\x00\x00\x00\x01] last=Optional[1\x00\x00\x00\x01] 2023-11-28 03:55:38,148 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c81e728d9d4c2f636f067f89cc14862c20231128/family1/f5109c7bb85f4140934dde7ba2134ab0 first=Optional[2\x00\x00\x00\x01] last=Optional[2\x00\x00\x00\x01] 2023-11-28 03:55:38,149 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=ns:testMinorCompaction,1,1701143722033.227f7071ca3e7b6612f5a149eeb75e98., hostname=jenkins-hbase4.apache.org,41203,1701143600653, seqNum=2 for row 1 with hfile group [{family1,hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c4ca4238a0b923820dcc509a6f75849b20231128/family1/6b09e18509fd4561a48a1a2828f2a4b3}] 2023-11-28 03:55:38,149 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=ns:testMinorCompaction,2,1701143722033.d38dce7d1890ef61d80a87d7305594c5., hostname=jenkins-hbase4.apache.org,41203,1701143600653, seqNum=2 for row 2 with hfile group [{family1,hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c81e728d9d4c2f636f067f89cc14862c20231128/family1/f5109c7bb85f4140934dde7ba2134ab0}] 2023-11-28 03:55:38,151 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41203] regionserver.HStore(763): Validating hfile at hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c81e728d9d4c2f636f067f89cc14862c20231128/family1/f5109c7bb85f4140934dde7ba2134ab0 for inclusion in d38dce7d1890ef61d80a87d7305594c5/family1 2023-11-28 03:55:38,151 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HStore(763): Validating hfile at hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c4ca4238a0b923820dcc509a6f75849b20231128/family1/6b09e18509fd4561a48a1a2828f2a4b3 for inclusion in 227f7071ca3e7b6612f5a149eeb75e98/family1 2023-11-28 03:55:38,153 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41203] regionserver.HStore(775): HFile bounds: first=2\x00\x00\x00\x01 last=2\x00\x00\x00\x01 2023-11-28 03:55:38,153 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41203] regionserver.HStore(777): Region bounds: first=2 last= 2023-11-28 03:55:38,154 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HStore(775): HFile bounds: first=1\x00\x00\x00\x01 last=1\x00\x00\x00\x01 2023-11-28 03:55:38,154 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41203] regionserver.HRegion(2446): Flush status journal for d38dce7d1890ef61d80a87d7305594c5: 2023-11-28 03:55:38,154 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HStore(777): Region bounds: first=1 last=2 2023-11-28 03:55:38,154 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HRegion(2446): Flush status journal for 227f7071ca3e7b6612f5a149eeb75e98: 2023-11-28 03:55:38,154 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41203] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c81e728d9d4c2f636f067f89cc14862c20231128/family1/f5109c7bb85f4140934dde7ba2134ab0 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__ns_testMinorCompaction__df7eboksb80isdfaerk0jt9ob2uu6e7a8bp2drk3jgtakcphdm2kehejcdisu71k/family1/f5109c7bb85f4140934dde7ba2134ab0 2023-11-28 03:55:38,154 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c4ca4238a0b923820dcc509a6f75849b20231128/family1/6b09e18509fd4561a48a1a2828f2a4b3 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__ns_testMinorCompaction__7rm78gtfh03gq844bic6d15eid94gqu5v7b4fdfke9m5hpn1vv2nmh3v2c5qo9aa/family1/6b09e18509fd4561a48a1a2828f2a4b3 2023-11-28 03:55:38,156 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41203] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__ns_testMinorCompaction__df7eboksb80isdfaerk0jt9ob2uu6e7a8bp2drk3jgtakcphdm2kehejcdisu71k/family1/f5109c7bb85f4140934dde7ba2134ab0 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/d38dce7d1890ef61d80a87d7305594c5/family1/e984b4f3f08744e0b8be0e013c0a5ada_SeqId_28_ 2023-11-28 03:55:38,157 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__ns_testMinorCompaction__7rm78gtfh03gq844bic6d15eid94gqu5v7b4fdfke9m5hpn1vv2nmh3v2c5qo9aa/family1/6b09e18509fd4561a48a1a2828f2a4b3 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/227f7071ca3e7b6612f5a149eeb75e98/family1/08a9aa81e8ce48fba4f50764d896b0aa_SeqId_28_ 2023-11-28 03:55:38,157 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41203] regionserver.HStore(849): Loaded HFile hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__ns_testMinorCompaction__df7eboksb80isdfaerk0jt9ob2uu6e7a8bp2drk3jgtakcphdm2kehejcdisu71k/family1/f5109c7bb85f4140934dde7ba2134ab0 into d38dce7d1890ef61d80a87d7305594c5/family1 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/d38dce7d1890ef61d80a87d7305594c5/family1/e984b4f3f08744e0b8be0e013c0a5ada_SeqId_28_ - updating store file list. 2023-11-28 03:55:38,157 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HStore(849): Loaded HFile hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__ns_testMinorCompaction__7rm78gtfh03gq844bic6d15eid94gqu5v7b4fdfke9m5hpn1vv2nmh3v2c5qo9aa/family1/6b09e18509fd4561a48a1a2828f2a4b3 into 227f7071ca3e7b6612f5a149eeb75e98/family1 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/227f7071ca3e7b6612f5a149eeb75e98/family1/08a9aa81e8ce48fba4f50764d896b0aa_SeqId_28_ - updating store file list. 2023-11-28 03:55:38,160 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41203] regionserver.HStore(882): Loaded HFile hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/d38dce7d1890ef61d80a87d7305594c5/family1/e984b4f3f08744e0b8be0e013c0a5ada_SeqId_28_ into d38dce7d1890ef61d80a87d7305594c5/family1 2023-11-28 03:55:38,160 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41203] regionserver.HStore(855): Successfully loaded hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__ns_testMinorCompaction__df7eboksb80isdfaerk0jt9ob2uu6e7a8bp2drk3jgtakcphdm2kehejcdisu71k/family1/f5109c7bb85f4140934dde7ba2134ab0 into d38dce7d1890ef61d80a87d7305594c5/family1 (new location: hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/d38dce7d1890ef61d80a87d7305594c5/family1/e984b4f3f08744e0b8be0e013c0a5ada_SeqId_28_) 2023-11-28 03:55:38,160 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HStore(882): Loaded HFile hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/227f7071ca3e7b6612f5a149eeb75e98/family1/08a9aa81e8ce48fba4f50764d896b0aa_SeqId_28_ into 227f7071ca3e7b6612f5a149eeb75e98/family1 2023-11-28 03:55:38,160 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.HStore(855): Successfully loaded hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__ns_testMinorCompaction__7rm78gtfh03gq844bic6d15eid94gqu5v7b4fdfke9m5hpn1vv2nmh3v2c5qo9aa/family1/6b09e18509fd4561a48a1a2828f2a4b3 into 227f7071ca3e7b6612f5a149eeb75e98/family1 (new location: hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/227f7071ca3e7b6612f5a149eeb75e98/family1/08a9aa81e8ce48fba4f50764d896b0aa_SeqId_28_) 2023-11-28 03:55:38,160 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41203] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__ns_testMinorCompaction__df7eboksb80isdfaerk0jt9ob2uu6e7a8bp2drk3jgtakcphdm2kehejcdisu71k/family1/f5109c7bb85f4140934dde7ba2134ab0 2023-11-28 03:55:38,161 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41203] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__ns_testMinorCompaction__7rm78gtfh03gq844bic6d15eid94gqu5v7b4fdfke9m5hpn1vv2nmh3v2c5qo9aa/family1/6b09e18509fd4561a48a1a2828f2a4b3 2023-11-28 03:55:38,162 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20231128 table='ns:testMinorCompaction' column='family1' 2023-11-28 03:55:38,162 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20231128 table='ns:testMinorCompaction' column='family1' 2023-11-28 03:55:38,163 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(360): Archiving compacted files. 2023-11-28 03:55:38,163 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(360): Archiving compacted files. 2023-11-28 03:55:38,164 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c202311284c388f1af8be4ade99ec99cac0274a00 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c202311284c388f1af8be4ade99ec99cac0274a00 2023-11-28 03:55:38,164 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(508): Table ns:testMinorCompaction 1 expired mob files are deleted 2023-11-28 03:55:38,164 DEBUG [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c81e728d9d4c2f636f067f89cc14862c20231128' 2023-11-28 03:55:38,164 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b202311287e09242846d741afb55ff45c59b2dc73 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b202311287e09242846d741afb55ff45c59b2dc73 2023-11-28 03:55:38,164 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(508): Table ns:testMinorCompaction 1 expired mob files are deleted 2023-11-28 03:55:38,164 DEBUG [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c4ca4238a0b923820dcc509a6f75849b20231128' 2023-11-28 03:55:38,165 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=c81e728d9d4c2f636f067f89cc14862c20231128 for table='ns:testMinorCompaction' and column='family1' 2023-11-28 03:55:38,165 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=c4ca4238a0b923820dcc509a6f75849b20231128 for table='ns:testMinorCompaction' and column='family1' 2023-11-28 03:55:38,165 INFO [Listener at localhost/42853] compactions.PartitionedMobCompactor(354): After compaction, there are 3 mob files. table='ns:testMinorCompaction' column='family1' 2023-11-28 03:55:38,167 DEBUG [Listener at localhost/42853] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-11-28 03:55:38,169 DEBUG [Listener at localhost/42853] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-11-28 03:55:38,172 DEBUG [Listener at localhost/42853] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-11-28 03:55:38,173 DEBUG [Listener at localhost/42853] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-11-28 03:55:38,188 INFO [Listener at localhost/42853] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMinorCompaction Thread=254 (was 286), OpenFileDescriptor=673 (was 771), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=174 (was 196), ProcessCount=168 (was 168), AvailableMemoryMB=6620 (was 7034) 2023-11-28 03:55:38,199 INFO [Listener at localhost/42853] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMinorCompactionWithMonthlyPolicy Thread=254, OpenFileDescriptor=673, MaxFileDescriptor=60000, SystemLoadAverage=174, ProcessCount=168, AvailableMemoryMB=6620 2023-11-28 03:55:38,200 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testMinorCompactionWithMonthlyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} 2023-11-28 03:55:38,201 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] procedure2.ProcedureExecutor(1028): Stored pid=112, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:38,202 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_PRE_OPERATION 2023-11-28 03:55:38,202 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] master.MasterRpcServices(700): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMinorCompactionWithMonthlyPolicy" procId is: 112 2023-11-28 03:55:38,203 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-11-28 03:55:38,203 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] master.MasterRpcServices(1230): Checking to see if procedure is done pid=112 2023-11-28 03:55:38,204 DEBUG [HFileArchiver-19] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/.tmp/data/default/testMinorCompactionWithMonthlyPolicy/2be0907b8bc55af7a11c257a83fc88d6 2023-11-28 03:55:38,204 DEBUG [HFileArchiver-19] backup.HFileArchiver(153): Directory hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/.tmp/data/default/testMinorCompactionWithMonthlyPolicy/2be0907b8bc55af7a11c257a83fc88d6 empty. 2023-11-28 03:55:38,205 DEBUG [HFileArchiver-19] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/.tmp/data/default/testMinorCompactionWithMonthlyPolicy/2be0907b8bc55af7a11c257a83fc88d6 2023-11-28 03:55:38,205 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(328): Archived testMinorCompactionWithMonthlyPolicy regions 2023-11-28 03:55:38,304 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] master.MasterRpcServices(1230): Checking to see if procedure is done pid=112 2023-11-28 03:55:38,505 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] master.MasterRpcServices(1230): Checking to see if procedure is done pid=112 2023-11-28 03:55:38,611 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/.tmp/data/default/testMinorCompactionWithMonthlyPolicy/.tabledesc/.tableinfo.0000000001 2023-11-28 03:55:38,613 INFO [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(7675): creating {ENCODED => 2be0907b8bc55af7a11c257a83fc88d6, NAME => 'testMinorCompactionWithMonthlyPolicy,,1701143738200.2be0907b8bc55af7a11c257a83fc88d6.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMinorCompactionWithMonthlyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}, regionDir=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/.tmp 2023-11-28 03:55:38,806 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] master.MasterRpcServices(1230): Checking to see if procedure is done pid=112 2023-11-28 03:55:39,018 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(866): Instantiated testMinorCompactionWithMonthlyPolicy,,1701143738200.2be0907b8bc55af7a11c257a83fc88d6.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-28 03:55:39,018 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1604): Closing 2be0907b8bc55af7a11c257a83fc88d6, disabling compactions & flushes 2023-11-28 03:55:39,018 INFO [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1626): Closing region testMinorCompactionWithMonthlyPolicy,,1701143738200.2be0907b8bc55af7a11c257a83fc88d6. 2023-11-28 03:55:39,018 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMinorCompactionWithMonthlyPolicy,,1701143738200.2be0907b8bc55af7a11c257a83fc88d6. 2023-11-28 03:55:39,018 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1714): Acquired close lock on testMinorCompactionWithMonthlyPolicy,,1701143738200.2be0907b8bc55af7a11c257a83fc88d6. after waiting 0 ms 2023-11-28 03:55:39,018 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1724): Updates disabled for region testMinorCompactionWithMonthlyPolicy,,1701143738200.2be0907b8bc55af7a11c257a83fc88d6. 2023-11-28 03:55:39,018 INFO [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1838): Closed testMinorCompactionWithMonthlyPolicy,,1701143738200.2be0907b8bc55af7a11c257a83fc88d6. 2023-11-28 03:55:39,018 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1558): Region close journal for 2be0907b8bc55af7a11c257a83fc88d6: 2023-11-28 03:55:39,020 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_ADD_TO_META 2023-11-28 03:55:39,021 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMinorCompactionWithMonthlyPolicy,,1701143738200.2be0907b8bc55af7a11c257a83fc88d6.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1701143739021"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1701143739021"}]},"ts":"1701143739021"} 2023-11-28 03:55:39,023 INFO [PEWorker-4] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-11-28 03:55:39,024 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-11-28 03:55:39,024 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMinorCompactionWithMonthlyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1701143739024"}]},"ts":"1701143739024"} 2023-11-28 03:55:39,025 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testMinorCompactionWithMonthlyPolicy, state=ENABLING in hbase:meta 2023-11-28 03:55:39,034 INFO [PEWorker-4] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMinorCompactionWithMonthlyPolicy, region=2be0907b8bc55af7a11c257a83fc88d6, ASSIGN}] 2023-11-28 03:55:39,042 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMinorCompactionWithMonthlyPolicy, region=2be0907b8bc55af7a11c257a83fc88d6, ASSIGN 2023-11-28 03:55:39,043 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMinorCompactionWithMonthlyPolicy, region=2be0907b8bc55af7a11c257a83fc88d6, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,41203,1701143600653; forceNewPlan=false, retain=false 2023-11-28 03:55:39,194 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=113 updating hbase:meta row=2be0907b8bc55af7a11c257a83fc88d6, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:39,194 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMinorCompactionWithMonthlyPolicy,,1701143738200.2be0907b8bc55af7a11c257a83fc88d6.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1701143739194"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1701143739194"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1701143739194"}]},"ts":"1701143739194"} 2023-11-28 03:55:39,196 INFO [PEWorker-5] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=114, ppid=113, state=RUNNABLE; OpenRegionProcedure 2be0907b8bc55af7a11c257a83fc88d6, server=jenkins-hbase4.apache.org,41203,1701143600653}] 2023-11-28 03:55:39,307 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] master.MasterRpcServices(1230): Checking to see if procedure is done pid=112 2023-11-28 03:55:39,347 DEBUG [RSProcedureDispatcher-pool-1] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:39,349 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testMinorCompactionWithMonthlyPolicy,,1701143738200.2be0907b8bc55af7a11c257a83fc88d6. 2023-11-28 03:55:39,350 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 2be0907b8bc55af7a11c257a83fc88d6, NAME => 'testMinorCompactionWithMonthlyPolicy,,1701143738200.2be0907b8bc55af7a11c257a83fc88d6.', STARTKEY => '', ENDKEY => ''} 2023-11-28 03:55:39,350 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMinorCompactionWithMonthlyPolicy 2be0907b8bc55af7a11c257a83fc88d6 2023-11-28 03:55:39,350 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMinorCompactionWithMonthlyPolicy,,1701143738200.2be0907b8bc55af7a11c257a83fc88d6.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-28 03:55:39,350 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 2be0907b8bc55af7a11c257a83fc88d6 2023-11-28 03:55:39,350 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 2be0907b8bc55af7a11c257a83fc88d6 2023-11-28 03:55:39,351 INFO [StoreOpener-2be0907b8bc55af7a11c257a83fc88d6-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 2be0907b8bc55af7a11c257a83fc88d6 2023-11-28 03:55:39,352 DEBUG [StoreOpener-2be0907b8bc55af7a11c257a83fc88d6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithMonthlyPolicy/2be0907b8bc55af7a11c257a83fc88d6/family1 2023-11-28 03:55:39,352 DEBUG [StoreOpener-2be0907b8bc55af7a11c257a83fc88d6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithMonthlyPolicy/2be0907b8bc55af7a11c257a83fc88d6/family1 2023-11-28 03:55:39,352 INFO [StoreOpener-2be0907b8bc55af7a11c257a83fc88d6-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 2be0907b8bc55af7a11c257a83fc88d6 columnFamilyName family1 2023-11-28 03:55:39,354 INFO [StoreOpener-2be0907b8bc55af7a11c257a83fc88d6-1] regionserver.HStore(310): Store=2be0907b8bc55af7a11c257a83fc88d6/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-28 03:55:39,354 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithMonthlyPolicy/2be0907b8bc55af7a11c257a83fc88d6 2023-11-28 03:55:39,355 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithMonthlyPolicy/2be0907b8bc55af7a11c257a83fc88d6 2023-11-28 03:55:39,356 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 2be0907b8bc55af7a11c257a83fc88d6 2023-11-28 03:55:39,358 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithMonthlyPolicy/2be0907b8bc55af7a11c257a83fc88d6/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-11-28 03:55:39,359 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 2be0907b8bc55af7a11c257a83fc88d6; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=65054179, jitterRate=-0.03061719238758087}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-11-28 03:55:39,359 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 2be0907b8bc55af7a11c257a83fc88d6: 2023-11-28 03:55:39,359 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2339): Post open deploy tasks for testMinorCompactionWithMonthlyPolicy,,1701143738200.2be0907b8bc55af7a11c257a83fc88d6., pid=114, masterSystemTime=1701143739347 2023-11-28 03:55:39,360 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2366): Finished post open deploy task for testMinorCompactionWithMonthlyPolicy,,1701143738200.2be0907b8bc55af7a11c257a83fc88d6. 2023-11-28 03:55:39,360 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testMinorCompactionWithMonthlyPolicy,,1701143738200.2be0907b8bc55af7a11c257a83fc88d6. 2023-11-28 03:55:39,361 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=113 updating hbase:meta row=2be0907b8bc55af7a11c257a83fc88d6, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:39,361 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMinorCompactionWithMonthlyPolicy,,1701143738200.2be0907b8bc55af7a11c257a83fc88d6.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1701143739361"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1701143739361"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1701143739361"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1701143739361"}]},"ts":"1701143739361"} 2023-11-28 03:55:39,363 INFO [PEWorker-3] procedure2.ProcedureExecutor(1825): Finished subprocedure pid=114, resume processing ppid=113 2023-11-28 03:55:39,363 INFO [PEWorker-3] procedure2.ProcedureExecutor(1409): Finished pid=114, ppid=113, state=SUCCESS; OpenRegionProcedure 2be0907b8bc55af7a11c257a83fc88d6, server=jenkins-hbase4.apache.org,41203,1701143600653 in 166 msec 2023-11-28 03:55:39,364 INFO [PEWorker-4] procedure2.ProcedureExecutor(1825): Finished subprocedure pid=113, resume processing ppid=112 2023-11-28 03:55:39,364 INFO [PEWorker-4] procedure2.ProcedureExecutor(1409): Finished pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testMinorCompactionWithMonthlyPolicy, region=2be0907b8bc55af7a11c257a83fc88d6, ASSIGN in 329 msec 2023-11-28 03:55:39,365 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-11-28 03:55:39,365 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMinorCompactionWithMonthlyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1701143739365"}]},"ts":"1701143739365"} 2023-11-28 03:55:39,366 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testMinorCompactionWithMonthlyPolicy, state=ENABLED in hbase:meta 2023-11-28 03:55:39,368 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_POST_OPERATION 2023-11-28 03:55:39,369 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=112, state=SUCCESS; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy in 1.1680 sec 2023-11-28 03:55:40,308 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] master.MasterRpcServices(1230): Checking to see if procedure is done pid=112 2023-11-28 03:55:40,308 INFO [Listener at localhost/42853] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMinorCompactionWithMonthlyPolicy, procId: 112 completed 2023-11-28 03:55:40,330 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-28 03:55:40,340 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:40,340 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-11-28 03:55:40,340 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:55:40,340 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-11-28 03:55:40,340 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-11-28 03:55:40,341 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:40,341 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:40,342 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:55:40,342 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:40,342 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:55:40,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:55:40,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:40,343 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:40,343 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:40,343 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-28 03:55:40,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-28 03:55:40,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:40,344 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:40,344 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:40,344 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-11-28 03:55:40,344 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:55:40,344 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-11-28 03:55:40,348 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-28 03:55:40,348 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-28 03:55:40,348 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1701143738200.2be0907b8bc55af7a11c257a83fc88d6. 2023-11-28 03:55:40,348 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1701143738200.2be0907b8bc55af7a11c257a83fc88d6. started... 2023-11-28 03:55:40,349 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 2be0907b8bc55af7a11c257a83fc88d6 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-11-28 03:55:40,396 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMinorCompactionWithMonthlyPolicy' 2023-11-28 03:55:40,774 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015113033df26fab23641fbaeaa982ab67fc807 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2015113033df26fab23641fbaeaa982ab67fc807 2023-11-28 03:55:41,177 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithMonthlyPolicy/2be0907b8bc55af7a11c257a83fc88d6/.tmp/family1/cea064f5d1454115a914f776a9041031 2023-11-28 03:55:41,182 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithMonthlyPolicy/2be0907b8bc55af7a11c257a83fc88d6/.tmp/family1/cea064f5d1454115a914f776a9041031 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithMonthlyPolicy/2be0907b8bc55af7a11c257a83fc88d6/family1/cea064f5d1454115a914f776a9041031 2023-11-28 03:55:41,186 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithMonthlyPolicy/2be0907b8bc55af7a11c257a83fc88d6/family1/cea064f5d1454115a914f776a9041031, entries=1000, sequenceid=5, filesize=166.0 K 2023-11-28 03:55:41,187 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for 2be0907b8bc55af7a11c257a83fc88d6 in 838ms, sequenceid=5, compaction requested=false 2023-11-28 03:55:41,188 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 2be0907b8bc55af7a11c257a83fc88d6: 2023-11-28 03:55:41,188 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1701143738200.2be0907b8bc55af7a11c257a83fc88d6. 2023-11-28 03:55:41,188 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-28 03:55:41,188 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-28 03:55:41,188 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:41,188 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-11-28 03:55:41,188 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-11-28 03:55:41,193 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:41,193 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:41,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:41,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:55:41,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:55:41,194 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:41,194 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-28 03:55:41,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:55:41,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:55:41,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:41,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:41,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:55:41,195 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-11-28 03:55:41,195 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6e99f541[Count = 0] remaining members to acquire global barrier 2023-11-28 03:55:41,195 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-11-28 03:55:41,195 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:41,196 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:41,197 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:41,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:41,197 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-11-28 03:55:41,197 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-11-28 03:55:41,197 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,41203,1701143600653' in zk 2023-11-28 03:55:41,197 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:41,197 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-28 03:55:41,198 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-11-28 03:55:41,198 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:55:41,198 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:41,198 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-11-28 03:55:41,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:41,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:55:41,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:55:41,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:55:41,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:55:41,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:41,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:41,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:55:41,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:41,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:41,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,41203,1701143600653': 2023-11-28 03:55:41,201 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-11-28 03:55:41,201 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-11-28 03:55:41,201 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-28 03:55:41,201 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-28 03:55:41,201 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:41,201 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-28 03:55:41,204 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:41,204 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:41,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:41,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:55:41,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:55:41,204 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:41,204 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:55:41,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:41,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:55:41,204 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:41,205 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:55:41,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:55:41,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:41,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:41,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:55:41,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:41,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:41,206 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:41,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:55:41,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:41,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:41,208 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:41,209 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:41,209 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:41,209 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:55:41,209 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:55:41,209 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:55:41,209 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:41,209 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:55:41,209 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:41,209 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-11-28 03:55:41,209 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-28 03:55:41,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:55:41,209 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:41,209 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:41,209 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-11-28 03:55:41,209 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-28 03:55:41,210 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:55:41,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:55:41,310 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-28 03:55:41,310 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-28 03:55:41,314 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-28 03:55:41,316 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-11-28 03:55:41,316 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:41,316 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-11-28 03:55:41,316 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:55:41,316 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-11-28 03:55:41,316 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-11-28 03:55:41,317 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:41,317 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:41,318 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:41,318 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:55:41,318 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:55:41,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:55:41,318 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:41,318 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-28 03:55:41,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:41,319 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:41,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-28 03:55:41,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:41,319 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:41,319 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-11-28 03:55:41,319 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:41,320 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-11-28 03:55:41,320 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:55:41,320 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-11-28 03:55:41,320 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-28 03:55:41,320 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-28 03:55:41,320 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1701143738200.2be0907b8bc55af7a11c257a83fc88d6. 2023-11-28 03:55:41,320 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1701143738200.2be0907b8bc55af7a11c257a83fc88d6. started... 2023-11-28 03:55:41,320 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 2be0907b8bc55af7a11c257a83fc88d6 1/1 column families, dataSize=74 B heapSize=408 B 2023-11-28 03:55:41,731 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128d976405866414a7c90816707e92bbf61 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151128d976405866414a7c90816707e92bbf61 2023-11-28 03:55:42,136 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithMonthlyPolicy/2be0907b8bc55af7a11c257a83fc88d6/.tmp/family1/c6fd27a0897f4d32bfd04e186a6a88d7 2023-11-28 03:55:42,141 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithMonthlyPolicy/2be0907b8bc55af7a11c257a83fc88d6/.tmp/family1/c6fd27a0897f4d32bfd04e186a6a88d7 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithMonthlyPolicy/2be0907b8bc55af7a11c257a83fc88d6/family1/c6fd27a0897f4d32bfd04e186a6a88d7 2023-11-28 03:55:42,144 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithMonthlyPolicy/2be0907b8bc55af7a11c257a83fc88d6/family1/c6fd27a0897f4d32bfd04e186a6a88d7, entries=1, sequenceid=9, filesize=5.0 K 2023-11-28 03:55:42,145 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for 2be0907b8bc55af7a11c257a83fc88d6 in 825ms, sequenceid=9, compaction requested=false 2023-11-28 03:55:42,145 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 2be0907b8bc55af7a11c257a83fc88d6: 2023-11-28 03:55:42,145 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1701143738200.2be0907b8bc55af7a11c257a83fc88d6. 2023-11-28 03:55:42,145 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-28 03:55:42,145 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-28 03:55:42,146 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:42,146 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-11-28 03:55:42,146 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-11-28 03:55:42,147 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:42,147 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:42,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:42,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:55:42,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:55:42,148 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:42,148 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-28 03:55:42,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:55:42,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:55:42,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:42,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:42,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:55:42,149 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-11-28 03:55:42,149 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@70cadee1[Count = 0] remaining members to acquire global barrier 2023-11-28 03:55:42,149 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-11-28 03:55:42,149 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:42,151 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:42,151 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:42,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:42,151 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-11-28 03:55:42,152 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-11-28 03:55:42,152 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,41203,1701143600653' in zk 2023-11-28 03:55:42,152 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:42,152 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-28 03:55:42,153 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-11-28 03:55:42,153 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:55:42,153 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:42,153 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-11-28 03:55:42,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:42,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:55:42,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:55:42,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:55:42,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:55:42,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:42,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:42,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:55:42,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:42,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:42,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,41203,1701143600653': 2023-11-28 03:55:42,156 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-11-28 03:55:42,156 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-11-28 03:55:42,156 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-28 03:55:42,156 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-28 03:55:42,156 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:42,156 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-28 03:55:42,158 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:42,158 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:42,158 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:42,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:42,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:55:42,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:55:42,158 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:55:42,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:42,159 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:55:42,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:55:42,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:55:42,159 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:42,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:42,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:42,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:55:42,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:42,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:42,160 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:42,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:55:42,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:42,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:42,166 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:55:42,166 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:42,166 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:55:42,166 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:42,166 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:42,166 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:55:42,166 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-11-28 03:55:42,166 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-28 03:55:42,166 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:42,166 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:55:42,166 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:42,166 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:42,166 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:42,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:55:42,166 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-11-28 03:55:42,167 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-28 03:55:42,167 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:55:42,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:55:42,267 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-28 03:55:42,268 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-28 03:55:42,271 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-28 03:55:42,273 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-11-28 03:55:42,273 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:42,273 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-11-28 03:55:42,273 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:55:42,273 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-11-28 03:55:42,273 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-11-28 03:55:42,275 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:42,275 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:42,278 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:55:42,278 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:42,278 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:55:42,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:55:42,278 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:42,278 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-28 03:55:42,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:42,279 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:42,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-28 03:55:42,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:42,279 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:42,279 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-11-28 03:55:42,279 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:42,279 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-11-28 03:55:42,279 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:55:42,279 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-11-28 03:55:42,280 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-28 03:55:42,280 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-28 03:55:42,280 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1701143738200.2be0907b8bc55af7a11c257a83fc88d6. 2023-11-28 03:55:42,280 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1701143738200.2be0907b8bc55af7a11c257a83fc88d6. started... 2023-11-28 03:55:42,280 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 2be0907b8bc55af7a11c257a83fc88d6 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-28 03:55:42,694 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151201e203cd16e50e4e069ac46970dab945ac to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151201e203cd16e50e4e069ac46970dab945ac 2023-11-28 03:55:42,699 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithMonthlyPolicy/2be0907b8bc55af7a11c257a83fc88d6/.tmp/family1/3e9633614b064f288a2f874bdedccda4 2023-11-28 03:55:42,703 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithMonthlyPolicy/2be0907b8bc55af7a11c257a83fc88d6/.tmp/family1/3e9633614b064f288a2f874bdedccda4 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithMonthlyPolicy/2be0907b8bc55af7a11c257a83fc88d6/family1/3e9633614b064f288a2f874bdedccda4 2023-11-28 03:55:42,706 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithMonthlyPolicy/2be0907b8bc55af7a11c257a83fc88d6/family1/3e9633614b064f288a2f874bdedccda4, entries=1, sequenceid=13, filesize=4.9 K 2023-11-28 03:55:42,707 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 2be0907b8bc55af7a11c257a83fc88d6 in 427ms, sequenceid=13, compaction requested=true 2023-11-28 03:55:42,707 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 2be0907b8bc55af7a11c257a83fc88d6: 2023-11-28 03:55:42,708 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1701143738200.2be0907b8bc55af7a11c257a83fc88d6. 2023-11-28 03:55:42,708 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-28 03:55:42,708 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-28 03:55:42,708 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:42,708 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-11-28 03:55:42,708 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-11-28 03:55:42,709 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:42,709 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:42,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:42,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:55:42,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:55:42,710 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:42,710 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-28 03:55:42,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:55:42,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:55:42,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:42,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:42,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:55:42,711 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-11-28 03:55:42,712 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5813c1f5[Count = 0] remaining members to acquire global barrier 2023-11-28 03:55:42,712 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-11-28 03:55:42,712 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:42,713 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:42,713 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:42,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:42,713 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-11-28 03:55:42,713 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-11-28 03:55:42,713 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,41203,1701143600653' in zk 2023-11-28 03:55:42,713 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:42,713 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-28 03:55:42,714 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:42,714 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-11-28 03:55:42,714 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:55:42,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:42,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:55:42,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:55:42,714 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-11-28 03:55:42,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:55:42,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:55:42,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:42,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:42,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:55:42,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:42,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:42,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,41203,1701143600653': 2023-11-28 03:55:42,717 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-11-28 03:55:42,717 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-11-28 03:55:42,717 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-28 03:55:42,717 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-28 03:55:42,717 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:42,717 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-28 03:55:42,720 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:42,720 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:55:42,720 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:42,720 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:42,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:42,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:55:42,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:55:42,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:42,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:55:42,721 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:42,721 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:55:42,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:55:42,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:42,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:42,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:55:42,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:42,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:42,722 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:42,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:55:42,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:42,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:42,732 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:55:42,732 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:42,732 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:55:42,732 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:55:42,732 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:55:42,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:55:42,732 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-11-28 03:55:42,732 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-28 03:55:42,732 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:42,732 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:42,733 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:42,733 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:42,733 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:42,733 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:55:42,733 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-11-28 03:55:42,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:55:42,733 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:42,733 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-28 03:55:42,833 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-28 03:55:42,834 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-28 03:55:42,838 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-28 03:55:42,840 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-11-28 03:55:42,840 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:42,840 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-11-28 03:55:42,840 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:55:42,840 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-11-28 03:55:42,840 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-11-28 03:55:42,841 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:42,841 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:42,842 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:42,842 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:55:42,842 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:55:42,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:55:42,843 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:42,843 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-28 03:55:42,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:42,843 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:42,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-28 03:55:42,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:42,844 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:42,844 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-11-28 03:55:42,844 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:42,844 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-11-28 03:55:42,844 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:55:42,844 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-11-28 03:55:42,844 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-28 03:55:42,844 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-28 03:55:42,844 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1701143738200.2be0907b8bc55af7a11c257a83fc88d6. 2023-11-28 03:55:42,844 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1701143738200.2be0907b8bc55af7a11c257a83fc88d6. started... 2023-11-28 03:55:42,845 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 2be0907b8bc55af7a11c257a83fc88d6 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-28 03:55:42,859 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205eca9bb999a5a47dcbecec3189c97e23a to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151205eca9bb999a5a47dcbecec3189c97e23a 2023-11-28 03:55:43,268 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithMonthlyPolicy/2be0907b8bc55af7a11c257a83fc88d6/.tmp/family1/4c0587f025ab4f5cb8e544dec03eb416 2023-11-28 03:55:43,272 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithMonthlyPolicy/2be0907b8bc55af7a11c257a83fc88d6/.tmp/family1/4c0587f025ab4f5cb8e544dec03eb416 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithMonthlyPolicy/2be0907b8bc55af7a11c257a83fc88d6/family1/4c0587f025ab4f5cb8e544dec03eb416 2023-11-28 03:55:43,275 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithMonthlyPolicy/2be0907b8bc55af7a11c257a83fc88d6/family1/4c0587f025ab4f5cb8e544dec03eb416, entries=1, sequenceid=17, filesize=4.9 K 2023-11-28 03:55:43,275 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 2be0907b8bc55af7a11c257a83fc88d6 in 431ms, sequenceid=17, compaction requested=true 2023-11-28 03:55:43,276 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 2be0907b8bc55af7a11c257a83fc88d6: 2023-11-28 03:55:43,276 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1701143738200.2be0907b8bc55af7a11c257a83fc88d6. 2023-11-28 03:55:43,276 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-28 03:55:43,276 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-28 03:55:43,276 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:43,276 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-11-28 03:55:43,276 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-11-28 03:55:43,279 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:43,279 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:43,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:43,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:55:43,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:55:43,279 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:43,279 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-28 03:55:43,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:55:43,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:55:43,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:43,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:43,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:55:43,280 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-11-28 03:55:43,280 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1f2a7548[Count = 0] remaining members to acquire global barrier 2023-11-28 03:55:43,280 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-11-28 03:55:43,281 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:43,282 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:43,282 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:43,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:43,282 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-11-28 03:55:43,282 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-11-28 03:55:43,282 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,41203,1701143600653' in zk 2023-11-28 03:55:43,282 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:43,282 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-28 03:55:43,283 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-11-28 03:55:43,283 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:43,283 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:55:43,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:43,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:55:43,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:55:43,283 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-11-28 03:55:43,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:55:43,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:55:43,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:43,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:43,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:55:43,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:43,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:43,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,41203,1701143600653': 2023-11-28 03:55:43,286 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-11-28 03:55:43,286 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-11-28 03:55:43,286 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-28 03:55:43,286 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-28 03:55:43,286 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:43,286 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-28 03:55:43,288 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:43,288 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:43,288 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:43,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:43,288 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:55:43,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:43,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:55:43,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:55:43,288 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:55:43,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:55:43,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:55:43,289 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:43,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:43,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:43,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:55:43,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:43,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:43,290 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:43,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:55:43,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:43,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:43,294 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:43,294 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:55:43,294 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:43,294 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:55:43,294 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(399): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-11-28 03:55:43,294 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-11-28 03:55:43,294 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-28 03:55:43,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:55:43,294 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:55:43,294 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:43,294 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:55:43,294 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:43,294 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:43,294 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-11-28 03:55:43,294 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:55:43,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:55:43,294 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:43,294 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-28 03:55:43,294 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:43,394 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-28 03:55:43,395 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-28 03:55:43,399 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-28 03:55:43,400 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-11-28 03:55:43,401 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:43,401 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-11-28 03:55:43,401 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:55:43,401 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-11-28 03:55:43,401 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-11-28 03:55:43,401 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:43,401 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:43,403 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:43,403 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:55:43,403 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:55:43,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:55:43,403 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:43,403 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-28 03:55:43,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:43,403 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:43,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-28 03:55:43,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:43,404 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:43,404 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-11-28 03:55:43,404 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:43,404 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-11-28 03:55:43,404 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:55:43,404 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-11-28 03:55:43,405 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-28 03:55:43,405 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-28 03:55:43,405 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1701143738200.2be0907b8bc55af7a11c257a83fc88d6. 2023-11-28 03:55:43,405 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1701143738200.2be0907b8bc55af7a11c257a83fc88d6. started... 2023-11-28 03:55:43,405 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 2be0907b8bc55af7a11c257a83fc88d6 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-28 03:55:43,817 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512280e934a75fd004cd3b70891f20a5abfb5 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201512280e934a75fd004cd3b70891f20a5abfb5 2023-11-28 03:55:44,224 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithMonthlyPolicy/2be0907b8bc55af7a11c257a83fc88d6/.tmp/family1/923b8bdc2044471881fb741401865d7d 2023-11-28 03:55:44,228 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithMonthlyPolicy/2be0907b8bc55af7a11c257a83fc88d6/.tmp/family1/923b8bdc2044471881fb741401865d7d as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithMonthlyPolicy/2be0907b8bc55af7a11c257a83fc88d6/family1/923b8bdc2044471881fb741401865d7d 2023-11-28 03:55:44,231 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithMonthlyPolicy/2be0907b8bc55af7a11c257a83fc88d6/family1/923b8bdc2044471881fb741401865d7d, entries=1, sequenceid=21, filesize=4.9 K 2023-11-28 03:55:44,232 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 2be0907b8bc55af7a11c257a83fc88d6 in 827ms, sequenceid=21, compaction requested=true 2023-11-28 03:55:44,232 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 2be0907b8bc55af7a11c257a83fc88d6: 2023-11-28 03:55:44,232 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1701143738200.2be0907b8bc55af7a11c257a83fc88d6. 2023-11-28 03:55:44,232 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-28 03:55:44,232 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-28 03:55:44,232 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:44,232 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-11-28 03:55:44,232 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-11-28 03:55:44,234 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:44,234 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:44,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:44,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:55:44,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:55:44,234 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:44,235 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-28 03:55:44,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:55:44,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:55:44,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:44,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:44,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:55:44,236 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-11-28 03:55:44,236 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3eb8b811[Count = 0] remaining members to acquire global barrier 2023-11-28 03:55:44,236 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-11-28 03:55:44,236 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:44,237 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:44,237 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:44,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:44,237 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-11-28 03:55:44,237 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-11-28 03:55:44,237 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,41203,1701143600653' in zk 2023-11-28 03:55:44,237 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:44,237 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-28 03:55:44,239 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:44,239 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-11-28 03:55:44,239 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:55:44,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:44,239 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-11-28 03:55:44,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:55:44,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:55:44,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:55:44,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:55:44,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:44,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:44,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:55:44,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:44,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:44,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,41203,1701143600653': 2023-11-28 03:55:44,241 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-11-28 03:55:44,241 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-11-28 03:55:44,241 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-28 03:55:44,241 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-28 03:55:44,241 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:44,241 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-28 03:55:44,243 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:44,243 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:44,243 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:44,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:44,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:55:44,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:55:44,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:44,243 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:55:44,243 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:55:44,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:55:44,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:55:44,243 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:44,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:44,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:44,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:55:44,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:44,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:44,245 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:44,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:55:44,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:44,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:44,247 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:44,247 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:44,247 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:44,247 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:55:44,247 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:44,247 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-11-28 03:55:44,247 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-28 03:55:44,247 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:44,247 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:55:44,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:55:44,247 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:55:44,247 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:55:44,247 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:44,247 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-11-28 03:55:44,247 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:44,247 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-28 03:55:44,247 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:55:44,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:55:44,347 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-28 03:55:44,348 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-28 03:55:44,352 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-28 03:55:44,353 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-11-28 03:55:44,353 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:44,353 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-11-28 03:55:44,353 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:55:44,354 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-11-28 03:55:44,354 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-11-28 03:55:44,354 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:44,354 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:44,355 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:55:44,355 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:44,355 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:55:44,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:55:44,356 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:44,356 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-28 03:55:44,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:44,356 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:44,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-28 03:55:44,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:44,357 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:44,357 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-11-28 03:55:44,357 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:44,357 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-11-28 03:55:44,357 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:55:44,357 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-11-28 03:55:44,357 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-28 03:55:44,357 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-28 03:55:44,357 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1701143738200.2be0907b8bc55af7a11c257a83fc88d6. 2023-11-28 03:55:44,357 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1701143738200.2be0907b8bc55af7a11c257a83fc88d6. started... 2023-11-28 03:55:44,357 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 2be0907b8bc55af7a11c257a83fc88d6 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-28 03:55:44,769 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015123181c17102408c40c6ae505862695a59aa to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2015123181c17102408c40c6ae505862695a59aa 2023-11-28 03:55:44,887 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-11-28 03:55:45,174 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithMonthlyPolicy/2be0907b8bc55af7a11c257a83fc88d6/.tmp/family1/53b130e2ee5744618b0314f34850a4a9 2023-11-28 03:55:45,178 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithMonthlyPolicy/2be0907b8bc55af7a11c257a83fc88d6/.tmp/family1/53b130e2ee5744618b0314f34850a4a9 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithMonthlyPolicy/2be0907b8bc55af7a11c257a83fc88d6/family1/53b130e2ee5744618b0314f34850a4a9 2023-11-28 03:55:45,181 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithMonthlyPolicy/2be0907b8bc55af7a11c257a83fc88d6/family1/53b130e2ee5744618b0314f34850a4a9, entries=1, sequenceid=25, filesize=4.9 K 2023-11-28 03:55:45,182 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 2be0907b8bc55af7a11c257a83fc88d6 in 824ms, sequenceid=25, compaction requested=true 2023-11-28 03:55:45,182 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 2be0907b8bc55af7a11c257a83fc88d6: 2023-11-28 03:55:45,182 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1701143738200.2be0907b8bc55af7a11c257a83fc88d6. 2023-11-28 03:55:45,182 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-28 03:55:45,182 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-28 03:55:45,182 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:45,182 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-11-28 03:55:45,182 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-11-28 03:55:45,183 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:45,184 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:45,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:45,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:55:45,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:55:45,184 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:45,184 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-28 03:55:45,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:55:45,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:55:45,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:45,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:45,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:55:45,185 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-11-28 03:55:45,185 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@182344f1[Count = 0] remaining members to acquire global barrier 2023-11-28 03:55:45,185 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-11-28 03:55:45,186 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:45,187 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:45,187 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:45,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:45,187 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-11-28 03:55:45,187 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-11-28 03:55:45,187 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,41203,1701143600653' in zk 2023-11-28 03:55:45,187 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:45,187 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-28 03:55:45,188 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-11-28 03:55:45,188 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:55:45,188 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-11-28 03:55:45,189 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:45,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:45,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:55:45,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:55:45,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:55:45,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:55:45,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:45,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:45,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:55:45,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:45,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:45,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,41203,1701143600653': 2023-11-28 03:55:45,192 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-11-28 03:55:45,192 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-11-28 03:55:45,192 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-28 03:55:45,192 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-28 03:55:45,192 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:45,192 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-28 03:55:45,199 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:45,199 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:45,199 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:45,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:45,199 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:55:45,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:45,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:55:45,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:55:45,199 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:55:45,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:55:45,199 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:45,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:55:45,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:45,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:45,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:55:45,200 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:45,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:45,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:45,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:55:45,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:45,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:45,203 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:45,203 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:45,203 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:45,203 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:45,203 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:45,203 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:45,203 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:45,203 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:55:45,203 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:55:45,203 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-11-28 03:55:45,203 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-28 03:55:45,203 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:55:45,203 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(399): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-11-28 03:55:45,203 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:55:45,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:55:45,203 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-11-28 03:55:45,203 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-28 03:55:45,204 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:55:45,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:55:45,304 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-28 03:55:45,304 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-28 03:55:45,307 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-28 03:55:45,309 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-11-28 03:55:45,309 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:45,309 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-11-28 03:55:45,309 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:55:45,309 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-11-28 03:55:45,309 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-11-28 03:55:45,309 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:45,310 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:45,311 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:45,311 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:55:45,311 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:55:45,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:55:45,311 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:45,311 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-28 03:55:45,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:45,312 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:45,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-28 03:55:45,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:45,312 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:45,312 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-11-28 03:55:45,312 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:45,312 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-11-28 03:55:45,312 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:55:45,312 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-11-28 03:55:45,312 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-28 03:55:45,313 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-28 03:55:45,313 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1701143738200.2be0907b8bc55af7a11c257a83fc88d6. 2023-11-28 03:55:45,313 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1701143738200.2be0907b8bc55af7a11c257a83fc88d6. started... 2023-11-28 03:55:45,313 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 2be0907b8bc55af7a11c257a83fc88d6 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-28 03:55:45,723 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2016010191ab39a91ba948a99ea893e577c45280 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2016010191ab39a91ba948a99ea893e577c45280 2023-11-28 03:55:45,728 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithMonthlyPolicy/2be0907b8bc55af7a11c257a83fc88d6/.tmp/family1/fa0015c9744e42b09bf5e5496355a407 2023-11-28 03:55:45,732 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithMonthlyPolicy/2be0907b8bc55af7a11c257a83fc88d6/.tmp/family1/fa0015c9744e42b09bf5e5496355a407 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithMonthlyPolicy/2be0907b8bc55af7a11c257a83fc88d6/family1/fa0015c9744e42b09bf5e5496355a407 2023-11-28 03:55:45,736 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithMonthlyPolicy/2be0907b8bc55af7a11c257a83fc88d6/family1/fa0015c9744e42b09bf5e5496355a407, entries=1, sequenceid=29, filesize=4.9 K 2023-11-28 03:55:45,736 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 2be0907b8bc55af7a11c257a83fc88d6 in 423ms, sequenceid=29, compaction requested=true 2023-11-28 03:55:45,737 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 2be0907b8bc55af7a11c257a83fc88d6: 2023-11-28 03:55:45,737 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1701143738200.2be0907b8bc55af7a11c257a83fc88d6. 2023-11-28 03:55:45,737 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-28 03:55:45,737 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-28 03:55:45,737 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:45,737 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-11-28 03:55:45,737 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-11-28 03:55:45,738 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:45,738 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:45,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:45,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:55:45,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:55:45,739 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:45,739 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-28 03:55:45,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:55:45,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:55:45,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:45,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:45,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:55:45,740 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-11-28 03:55:45,740 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5e793269[Count = 0] remaining members to acquire global barrier 2023-11-28 03:55:45,740 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-11-28 03:55:45,740 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:45,742 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:45,742 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:45,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:45,742 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-11-28 03:55:45,742 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-11-28 03:55:45,742 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,41203,1701143600653' in zk 2023-11-28 03:55:45,742 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:45,743 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-28 03:55:45,744 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-11-28 03:55:45,744 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:45,744 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:55:45,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:45,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:55:45,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:55:45,744 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-11-28 03:55:45,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:55:45,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:55:45,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:45,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:45,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:55:45,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:45,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:45,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,41203,1701143600653': 2023-11-28 03:55:45,747 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-11-28 03:55:45,747 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-11-28 03:55:45,747 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-28 03:55:45,747 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-28 03:55:45,747 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:45,747 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-28 03:55:45,748 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:45,748 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:45,749 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:45,749 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:55:45,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:45,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:45,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:55:45,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:55:45,749 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:55:45,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:55:45,749 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:45,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:55:45,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:45,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:45,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:55:45,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:45,750 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:45,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:45,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:55:45,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:45,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:45,753 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:55:45,753 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:45,753 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:55:45,753 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:55:45,753 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:55:45,753 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(399): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-11-28 03:55:45,753 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-11-28 03:55:45,753 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-28 03:55:45,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:55:45,753 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:45,753 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:45,753 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:45,753 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:45,754 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:45,754 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:45,754 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-11-28 03:55:45,754 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:55:45,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:55:45,754 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-28 03:55:45,854 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-28 03:55:45,855 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-28 03:55:45,859 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-28 03:55:45,860 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-11-28 03:55:45,860 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:45,860 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-11-28 03:55:45,861 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:55:45,861 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-11-28 03:55:45,861 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-11-28 03:55:45,861 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:45,861 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:45,863 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:45,863 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:55:45,863 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:55:45,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:55:45,863 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:45,863 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-28 03:55:45,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:45,864 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:45,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-28 03:55:45,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:45,864 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:45,864 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-11-28 03:55:45,864 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:45,864 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-11-28 03:55:45,864 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:55:45,864 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-11-28 03:55:45,865 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-28 03:55:45,865 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-28 03:55:45,865 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1701143738200.2be0907b8bc55af7a11c257a83fc88d6. 2023-11-28 03:55:45,865 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1701143738200.2be0907b8bc55af7a11c257a83fc88d6. started... 2023-11-28 03:55:45,865 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 2be0907b8bc55af7a11c257a83fc88d6 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-28 03:55:46,276 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103be3d6da528e542e0b7a3d8b46a393f38 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20160103be3d6da528e542e0b7a3d8b46a393f38 2023-11-28 03:55:46,682 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithMonthlyPolicy/2be0907b8bc55af7a11c257a83fc88d6/.tmp/family1/9a93a6dbd8b54cf4bda37a3f617da20e 2023-11-28 03:55:46,686 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithMonthlyPolicy/2be0907b8bc55af7a11c257a83fc88d6/.tmp/family1/9a93a6dbd8b54cf4bda37a3f617da20e as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithMonthlyPolicy/2be0907b8bc55af7a11c257a83fc88d6/family1/9a93a6dbd8b54cf4bda37a3f617da20e 2023-11-28 03:55:46,689 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithMonthlyPolicy/2be0907b8bc55af7a11c257a83fc88d6/family1/9a93a6dbd8b54cf4bda37a3f617da20e, entries=1, sequenceid=33, filesize=4.9 K 2023-11-28 03:55:46,690 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 2be0907b8bc55af7a11c257a83fc88d6 in 825ms, sequenceid=33, compaction requested=true 2023-11-28 03:55:46,690 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 2be0907b8bc55af7a11c257a83fc88d6: 2023-11-28 03:55:46,690 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1701143738200.2be0907b8bc55af7a11c257a83fc88d6. 2023-11-28 03:55:46,690 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-28 03:55:46,690 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-28 03:55:46,690 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:46,691 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-11-28 03:55:46,691 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-11-28 03:55:46,692 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:46,692 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:46,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:46,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:55:46,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:55:46,693 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:46,693 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-28 03:55:46,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:55:46,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:55:46,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:46,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:46,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:55:46,694 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-11-28 03:55:46,694 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@12b58cdb[Count = 0] remaining members to acquire global barrier 2023-11-28 03:55:46,694 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-11-28 03:55:46,694 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:46,695 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:46,695 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:46,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:46,696 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-11-28 03:55:46,696 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-11-28 03:55:46,696 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,41203,1701143600653' in zk 2023-11-28 03:55:46,696 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:46,696 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-28 03:55:46,698 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:46,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:46,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:55:46,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:55:46,698 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-11-28 03:55:46,698 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:55:46,698 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-11-28 03:55:46,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:55:46,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:55:46,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:46,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:46,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:55:46,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:46,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:46,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,41203,1701143600653': 2023-11-28 03:55:46,700 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-11-28 03:55:46,700 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-11-28 03:55:46,701 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-28 03:55:46,701 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-28 03:55:46,701 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:46,701 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-28 03:55:46,702 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:46,702 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:46,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:46,702 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:46,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:55:46,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:55:46,702 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:55:46,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:46,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:55:46,703 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:46,703 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:55:46,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:55:46,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:46,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:46,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:55:46,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:46,704 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:46,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:46,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:55:46,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:46,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:46,706 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:46,706 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:55:46,706 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:46,706 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-11-28 03:55:46,707 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-28 03:55:46,706 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:55:46,706 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:46,707 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(399): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-11-28 03:55:46,706 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:55:46,706 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:55:46,707 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-11-28 03:55:46,707 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:46,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:55:46,707 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:46,707 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-28 03:55:46,707 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:46,707 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:46,707 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:55:46,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:55:46,807 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-28 03:55:46,808 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-28 03:55:46,812 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-28 03:55:46,814 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-11-28 03:55:46,814 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:46,814 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-11-28 03:55:46,814 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:55:46,814 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-11-28 03:55:46,814 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-11-28 03:55:46,814 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:46,815 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:46,816 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:55:46,816 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:55:46,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:55:46,816 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:46,817 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:46,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:46,817 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-28 03:55:46,817 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:46,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-28 03:55:46,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:46,817 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:46,818 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-11-28 03:55:46,818 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:46,818 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-11-28 03:55:46,818 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:55:46,818 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-11-28 03:55:46,818 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-28 03:55:46,818 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-28 03:55:46,818 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1701143738200.2be0907b8bc55af7a11c257a83fc88d6. 2023-11-28 03:55:46,818 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1701143738200.2be0907b8bc55af7a11c257a83fc88d6. started... 2023-11-28 03:55:46,818 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 2be0907b8bc55af7a11c257a83fc88d6 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-28 03:55:47,229 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20150907710561b37d454de78b0d7bf410e85eb5 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20150907710561b37d454de78b0d7bf410e85eb5 2023-11-28 03:55:47,634 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithMonthlyPolicy/2be0907b8bc55af7a11c257a83fc88d6/.tmp/family1/e64a147830fb4588836595d056ac5345 2023-11-28 03:55:47,638 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithMonthlyPolicy/2be0907b8bc55af7a11c257a83fc88d6/.tmp/family1/e64a147830fb4588836595d056ac5345 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithMonthlyPolicy/2be0907b8bc55af7a11c257a83fc88d6/family1/e64a147830fb4588836595d056ac5345 2023-11-28 03:55:47,641 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithMonthlyPolicy/2be0907b8bc55af7a11c257a83fc88d6/family1/e64a147830fb4588836595d056ac5345, entries=1, sequenceid=37, filesize=4.9 K 2023-11-28 03:55:47,641 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 2be0907b8bc55af7a11c257a83fc88d6 in 823ms, sequenceid=37, compaction requested=true 2023-11-28 03:55:47,641 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 2be0907b8bc55af7a11c257a83fc88d6: 2023-11-28 03:55:47,641 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1701143738200.2be0907b8bc55af7a11c257a83fc88d6. 2023-11-28 03:55:47,641 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-28 03:55:47,641 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-28 03:55:47,642 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:47,642 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-11-28 03:55:47,642 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-11-28 03:55:47,647 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:47,647 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:47,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:47,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:55:47,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:55:47,647 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:47,647 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-28 03:55:47,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:55:47,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:55:47,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:47,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:47,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:55:47,649 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-11-28 03:55:47,649 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@69b6c6ac[Count = 0] remaining members to acquire global barrier 2023-11-28 03:55:47,649 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-11-28 03:55:47,649 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:47,650 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:47,650 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:47,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:47,650 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-11-28 03:55:47,650 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-11-28 03:55:47,650 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,41203,1701143600653' in zk 2023-11-28 03:55:47,650 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:47,650 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-28 03:55:47,651 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:47,651 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-11-28 03:55:47,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:47,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:55:47,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:55:47,651 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:55:47,651 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-11-28 03:55:47,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:55:47,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:55:47,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:47,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:47,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:55:47,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:47,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:47,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,41203,1701143600653': 2023-11-28 03:55:47,653 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-11-28 03:55:47,653 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-11-28 03:55:47,653 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-28 03:55:47,654 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-28 03:55:47,654 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:47,654 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-28 03:55:47,655 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:47,655 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:47,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:47,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:55:47,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:55:47,655 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:47,655 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:55:47,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:47,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:55:47,656 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:55:47,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:55:47,656 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:47,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:47,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:47,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:55:47,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:47,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:47,657 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:47,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:55:47,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:47,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:47,660 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:47,660 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:55:47,660 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:47,660 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:55:47,660 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:47,660 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:55:47,660 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:55:47,660 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:47,660 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-11-28 03:55:47,660 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-28 03:55:47,660 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(399): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-11-28 03:55:47,660 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:47,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:55:47,660 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:47,660 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:47,660 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-11-28 03:55:47,660 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-28 03:55:47,660 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:55:47,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:55:47,761 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-28 03:55:47,761 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-28 03:55:47,764 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-28 03:55:47,766 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-11-28 03:55:47,766 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:47,766 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-11-28 03:55:47,766 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:55:47,766 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-11-28 03:55:47,766 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-11-28 03:55:47,766 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:47,766 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:47,768 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:47,768 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:55:47,768 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:55:47,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:55:47,768 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:47,768 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-28 03:55:47,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:47,769 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:47,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-28 03:55:47,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:47,769 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:47,769 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-11-28 03:55:47,769 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:47,769 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-11-28 03:55:47,769 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:55:47,769 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-11-28 03:55:47,769 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-28 03:55:47,769 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-28 03:55:47,770 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1701143738200.2be0907b8bc55af7a11c257a83fc88d6. 2023-11-28 03:55:47,770 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1701143738200.2be0907b8bc55af7a11c257a83fc88d6. started... 2023-11-28 03:55:47,770 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 2be0907b8bc55af7a11c257a83fc88d6 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-28 03:55:47,779 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151120f98f6bfd83834e9cb22f87abd0077929 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151120f98f6bfd83834e9cb22f87abd0077929 2023-11-28 03:55:48,183 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithMonthlyPolicy/2be0907b8bc55af7a11c257a83fc88d6/.tmp/family1/6d1551ff41ae4307bfb7c7ec99383e6b 2023-11-28 03:55:48,188 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithMonthlyPolicy/2be0907b8bc55af7a11c257a83fc88d6/.tmp/family1/6d1551ff41ae4307bfb7c7ec99383e6b as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithMonthlyPolicy/2be0907b8bc55af7a11c257a83fc88d6/family1/6d1551ff41ae4307bfb7c7ec99383e6b 2023-11-28 03:55:48,191 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithMonthlyPolicy/2be0907b8bc55af7a11c257a83fc88d6/family1/6d1551ff41ae4307bfb7c7ec99383e6b, entries=1, sequenceid=41, filesize=4.9 K 2023-11-28 03:55:48,192 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 2be0907b8bc55af7a11c257a83fc88d6 in 422ms, sequenceid=41, compaction requested=true 2023-11-28 03:55:48,192 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 2be0907b8bc55af7a11c257a83fc88d6: 2023-11-28 03:55:48,192 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1701143738200.2be0907b8bc55af7a11c257a83fc88d6. 2023-11-28 03:55:48,192 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-28 03:55:48,192 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-28 03:55:48,192 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:48,192 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-11-28 03:55:48,192 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-11-28 03:55:48,194 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:48,194 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:48,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:48,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:55:48,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:55:48,194 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:48,194 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-28 03:55:48,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:55:48,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:55:48,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:48,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:48,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:55:48,196 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-11-28 03:55:48,196 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6bb003a7[Count = 0] remaining members to acquire global barrier 2023-11-28 03:55:48,196 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-11-28 03:55:48,196 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:48,197 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:48,197 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:48,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:48,197 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-11-28 03:55:48,197 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-11-28 03:55:48,197 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,41203,1701143600653' in zk 2023-11-28 03:55:48,197 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:48,197 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-28 03:55:48,199 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:48,199 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-11-28 03:55:48,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:48,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:55:48,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:55:48,199 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:55:48,199 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-11-28 03:55:48,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:55:48,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:55:48,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:48,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:48,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:55:48,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:48,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:48,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,41203,1701143600653': 2023-11-28 03:55:48,201 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-11-28 03:55:48,201 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-11-28 03:55:48,201 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-28 03:55:48,201 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-28 03:55:48,202 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:48,202 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-28 03:55:48,203 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:48,203 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:48,203 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:48,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:48,203 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:55:48,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:48,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:55:48,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:55:48,203 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:55:48,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:55:48,203 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:48,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:55:48,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:48,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:48,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:55:48,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:48,205 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:48,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:48,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:55:48,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:48,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:48,207 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:48,207 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:48,207 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:48,207 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:48,207 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-11-28 03:55:48,207 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(399): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-11-28 03:55:48,207 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-28 03:55:48,207 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:48,207 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:55:48,207 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:55:48,207 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:48,207 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:55:48,207 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:48,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:55:48,207 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:55:48,207 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-11-28 03:55:48,207 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-28 03:55:48,207 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:55:48,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:55:48,307 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-28 03:55:48,308 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-28 03:55:48,309 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=38593] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}] of the table testMinorCompactionWithMonthlyPolicy 2023-11-28 03:55:48,310 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] procedure2.ProcedureExecutor(1028): Stored pid=115, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-11-28 03:55:48,311 DEBUG [PEWorker-5] locking.LockProcedure(309): LOCKED pid=115, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-11-28 03:55:48,312 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] mob.MobUtils(833): start MOB compaction of files for table='testMinorCompactionWithMonthlyPolicy', column='family1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-11-28 03:55:48,312 INFO [PEWorker-5] procedure2.TimeoutExecutorThread(81): ADDED pid=115, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1701144348312 2023-11-28 03:55:48,313 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-11-28 03:55:48,314 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMinorCompactionWithMonthlyPolicy' and column 'family1' 2023-11-28 03:55:48,314 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-11-28 03:55:48,315 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] zookeeper.ReadOnlyZKClient(139): Connect 0x2d3c06be to 127.0.0.1:55641 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-11-28 03:55:48,319 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] ipc.AbstractRpcClient(189): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@4d649a69, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-11-28 03:55:48,319 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-11-28 03:55:48,743 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103d57a860706e345d4b65f125320d93318 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20160103d57a860706e345d4b65f125320d93318 2023-11-28 03:55:48,745 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20160101 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-11-28 03:55:48,759 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(122): Using SIMPLE authentication for service=ClientService, sasl=false 2023-11-28 03:55:48,761 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:38968, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-11-28 03:55:48,767 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(122): Using SIMPLE authentication for service=MasterService, sasl=false 2023-11-28 03:55:48,768 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:56672, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-11-28 03:55:48,778 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/c771f0d342c64787b1be3a117c578bbb first=Optional[r5] last=Optional[r6] 2023-11-28 03:55:48,779 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMinorCompactionWithMonthlyPolicy,,1701143738200.2be0907b8bc55af7a11c257a83fc88d6., hostname=jenkins-hbase4.apache.org,41203,1701143600653, seqNum=2 for row with hfile group [{family1,hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/c771f0d342c64787b1be3a117c578bbb}] 2023-11-28 03:55:48,783 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41203] regionserver.HStore(763): Validating hfile at hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/c771f0d342c64787b1be3a117c578bbb for inclusion in 2be0907b8bc55af7a11c257a83fc88d6/family1 2023-11-28 03:55:48,785 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41203] regionserver.HStore(775): HFile bounds: first=r5 last=r6 2023-11-28 03:55:48,785 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41203] regionserver.HStore(777): Region bounds: first= last= 2023-11-28 03:55:48,785 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41203] regionserver.HRegion(2446): Flush status journal for 2be0907b8bc55af7a11c257a83fc88d6: 2023-11-28 03:55:48,786 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41203] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/c771f0d342c64787b1be3a117c578bbb to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testMinorCompactionWithMonthlyPolicy__hp3midtodab70bgh94sarjothv2ji8hla889mg3a57h0njp16c8iqodb4j427nam/family1/c771f0d342c64787b1be3a117c578bbb 2023-11-28 03:55:48,788 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41203] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testMinorCompactionWithMonthlyPolicy__hp3midtodab70bgh94sarjothv2ji8hla889mg3a57h0njp16c8iqodb4j427nam/family1/c771f0d342c64787b1be3a117c578bbb as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithMonthlyPolicy/2be0907b8bc55af7a11c257a83fc88d6/family1/d25b7f9ccb1446858757be0ec90dc462_SeqId_44_ 2023-11-28 03:55:48,788 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41203] regionserver.HStore(849): Loaded HFile hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testMinorCompactionWithMonthlyPolicy__hp3midtodab70bgh94sarjothv2ji8hla889mg3a57h0njp16c8iqodb4j427nam/family1/c771f0d342c64787b1be3a117c578bbb into 2be0907b8bc55af7a11c257a83fc88d6/family1 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithMonthlyPolicy/2be0907b8bc55af7a11c257a83fc88d6/family1/d25b7f9ccb1446858757be0ec90dc462_SeqId_44_ - updating store file list. 2023-11-28 03:55:48,792 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41203] regionserver.HStore(882): Loaded HFile hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithMonthlyPolicy/2be0907b8bc55af7a11c257a83fc88d6/family1/d25b7f9ccb1446858757be0ec90dc462_SeqId_44_ into 2be0907b8bc55af7a11c257a83fc88d6/family1 2023-11-28 03:55:48,792 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41203] regionserver.HStore(855): Successfully loaded hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testMinorCompactionWithMonthlyPolicy__hp3midtodab70bgh94sarjothv2ji8hla889mg3a57h0njp16c8iqodb4j427nam/family1/c771f0d342c64787b1be3a117c578bbb into 2be0907b8bc55af7a11c257a83fc88d6/family1 (new location: hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithMonthlyPolicy/2be0907b8bc55af7a11c257a83fc88d6/family1/d25b7f9ccb1446858757be0ec90dc462_SeqId_44_) 2023-11-28 03:55:48,792 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41203] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testMinorCompactionWithMonthlyPolicy__hp3midtodab70bgh94sarjothv2ji8hla889mg3a57h0njp16c8iqodb4j427nam/family1/c771f0d342c64787b1be3a117c578bbb 2023-11-28 03:55:48,792 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41203] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-11-28 03:55:48,795 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41203] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 50903 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-11-28 03:55:48,795 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41203] regionserver.HStore(1912): 2be0907b8bc55af7a11c257a83fc88d6/family1 is initiating minor compaction 2023-11-28 03:55:48,795 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41203] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@45666544; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-28 03:55:48,795 INFO [RS:0;jenkins-hbase4:41203-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 2be0907b8bc55af7a11c257a83fc88d6/family1 in testMinorCompactionWithMonthlyPolicy,,1701143738200.2be0907b8bc55af7a11c257a83fc88d6. 2023-11-28 03:55:48,795 INFO [RS:0;jenkins-hbase4:41203-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithMonthlyPolicy/2be0907b8bc55af7a11c257a83fc88d6/family1/c6fd27a0897f4d32bfd04e186a6a88d7, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithMonthlyPolicy/2be0907b8bc55af7a11c257a83fc88d6/family1/3e9633614b064f288a2f874bdedccda4, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithMonthlyPolicy/2be0907b8bc55af7a11c257a83fc88d6/family1/4c0587f025ab4f5cb8e544dec03eb416, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithMonthlyPolicy/2be0907b8bc55af7a11c257a83fc88d6/family1/923b8bdc2044471881fb741401865d7d, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithMonthlyPolicy/2be0907b8bc55af7a11c257a83fc88d6/family1/53b130e2ee5744618b0314f34850a4a9, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithMonthlyPolicy/2be0907b8bc55af7a11c257a83fc88d6/family1/fa0015c9744e42b09bf5e5496355a407, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithMonthlyPolicy/2be0907b8bc55af7a11c257a83fc88d6/family1/9a93a6dbd8b54cf4bda37a3f617da20e, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithMonthlyPolicy/2be0907b8bc55af7a11c257a83fc88d6/family1/e64a147830fb4588836595d056ac5345, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithMonthlyPolicy/2be0907b8bc55af7a11c257a83fc88d6/family1/6d1551ff41ae4307bfb7c7ec99383e6b, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithMonthlyPolicy/2be0907b8bc55af7a11c257a83fc88d6/family1/d25b7f9ccb1446858757be0ec90dc462_SeqId_44_] into tmpdir=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithMonthlyPolicy/2be0907b8bc55af7a11c257a83fc88d6/.tmp, totalSize=49.7 K 2023-11-28 03:55:48,795 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] compactions.Compactor(207): Compacting c6fd27a0897f4d32bfd04e186a6a88d7, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-11-28 03:55:48,795 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] compactions.Compactor(207): Compacting 3e9633614b064f288a2f874bdedccda4, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=13 2023-11-28 03:55:48,795 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] compactions.Compactor(207): Compacting 4c0587f025ab4f5cb8e544dec03eb416, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=17 2023-11-28 03:55:48,796 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] compactions.Compactor(207): Compacting 923b8bdc2044471881fb741401865d7d, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=21 2023-11-28 03:55:48,796 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] compactions.Compactor(207): Compacting 53b130e2ee5744618b0314f34850a4a9, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=25 2023-11-28 03:55:48,796 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] compactions.Compactor(207): Compacting fa0015c9744e42b09bf5e5496355a407, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=29 2023-11-28 03:55:48,796 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] compactions.Compactor(207): Compacting 9a93a6dbd8b54cf4bda37a3f617da20e, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=33 2023-11-28 03:55:48,796 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] compactions.Compactor(207): Compacting e64a147830fb4588836595d056ac5345, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=37 2023-11-28 03:55:48,796 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] compactions.Compactor(207): Compacting 6d1551ff41ae4307bfb7c7ec99383e6b, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=41 2023-11-28 03:55:48,796 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] compactions.Compactor(207): Compacting d25b7f9ccb1446858757be0ec90dc462_SeqId_44_, keycount=2, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=45 2023-11-28 03:55:48,797 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20160101 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-11-28 03:55:48,797 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-28 03:55:48,801 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2016010191ab39a91ba948a99ea893e577c45280 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2016010191ab39a91ba948a99ea893e577c45280 2023-11-28 03:55:48,802 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20160103be3d6da528e542e0b7a3d8b46a393f38 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20160103be3d6da528e542e0b7a3d8b46a393f38 2023-11-28 03:55:48,802 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMinorCompactionWithMonthlyPolicy 2 expired mob files are deleted 2023-11-28 03:55:48,802 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101' 2023-11-28 03:55:48,803 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-11-28 03:55:48,803 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-11-28 03:55:48,830 INFO [RS:0;jenkins-hbase4:41203-shortCompactions-0] throttle.PressureAwareThroughputController(145): 2be0907b8bc55af7a11c257a83fc88d6#family1#compaction#165 average throughput is 0.68 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 57.69 MB/second 2023-11-28 03:55:49,253 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithMonthlyPolicy/2be0907b8bc55af7a11c257a83fc88d6/.tmp/family1/9ebe785ee7884a45ba3e3d803df703a1 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithMonthlyPolicy/2be0907b8bc55af7a11c257a83fc88d6/family1/9ebe785ee7884a45ba3e3d803df703a1 2023-11-28 03:55:49,270 INFO [RS:0;jenkins-hbase4:41203-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in 2be0907b8bc55af7a11c257a83fc88d6/family1 of 2be0907b8bc55af7a11c257a83fc88d6 into 9ebe785ee7884a45ba3e3d803df703a1(size=6.6 K), total size for store is 172.6 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-11-28 03:55:49,270 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 2be0907b8bc55af7a11c257a83fc88d6: 2023-11-28 03:55:49,270 INFO [RS:0;jenkins-hbase4:41203-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMinorCompactionWithMonthlyPolicy,,1701143738200.2be0907b8bc55af7a11c257a83fc88d6., storeName=family1, fileCount=10, fileSize=49.7 K (5.0 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 5.2 K), priority=2, time=1701143748795; duration=0sec 2023-11-28 03:55:49,270 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-28 03:55:49,297 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151231cc67bcec330c44c88c575522ddbe7f25 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151231cc67bcec330c44c88c575522ddbe7f25 2023-11-28 03:55:49,298 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151201 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-11-28 03:55:49,306 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithMonthlyPolicy/2be0907b8bc55af7a11c257a83fc88d6/family1/c6fd27a0897f4d32bfd04e186a6a88d7, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithMonthlyPolicy/2be0907b8bc55af7a11c257a83fc88d6/family1/3e9633614b064f288a2f874bdedccda4, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithMonthlyPolicy/2be0907b8bc55af7a11c257a83fc88d6/family1/4c0587f025ab4f5cb8e544dec03eb416, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithMonthlyPolicy/2be0907b8bc55af7a11c257a83fc88d6/family1/923b8bdc2044471881fb741401865d7d, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithMonthlyPolicy/2be0907b8bc55af7a11c257a83fc88d6/family1/53b130e2ee5744618b0314f34850a4a9, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithMonthlyPolicy/2be0907b8bc55af7a11c257a83fc88d6/family1/fa0015c9744e42b09bf5e5496355a407, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithMonthlyPolicy/2be0907b8bc55af7a11c257a83fc88d6/family1/9a93a6dbd8b54cf4bda37a3f617da20e, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithMonthlyPolicy/2be0907b8bc55af7a11c257a83fc88d6/family1/e64a147830fb4588836595d056ac5345, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithMonthlyPolicy/2be0907b8bc55af7a11c257a83fc88d6/family1/6d1551ff41ae4307bfb7c7ec99383e6b, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithMonthlyPolicy/2be0907b8bc55af7a11c257a83fc88d6/family1/d25b7f9ccb1446858757be0ec90dc462_SeqId_44_] to archive 2023-11-28 03:55:49,307 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-28 03:55:49,309 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithMonthlyPolicy/2be0907b8bc55af7a11c257a83fc88d6/family1/c6fd27a0897f4d32bfd04e186a6a88d7 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMinorCompactionWithMonthlyPolicy/2be0907b8bc55af7a11c257a83fc88d6/family1/c6fd27a0897f4d32bfd04e186a6a88d7 2023-11-28 03:55:49,310 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithMonthlyPolicy/2be0907b8bc55af7a11c257a83fc88d6/family1/3e9633614b064f288a2f874bdedccda4 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMinorCompactionWithMonthlyPolicy/2be0907b8bc55af7a11c257a83fc88d6/family1/3e9633614b064f288a2f874bdedccda4 2023-11-28 03:55:49,311 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithMonthlyPolicy/2be0907b8bc55af7a11c257a83fc88d6/family1/4c0587f025ab4f5cb8e544dec03eb416 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMinorCompactionWithMonthlyPolicy/2be0907b8bc55af7a11c257a83fc88d6/family1/4c0587f025ab4f5cb8e544dec03eb416 2023-11-28 03:55:49,313 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithMonthlyPolicy/2be0907b8bc55af7a11c257a83fc88d6/family1/923b8bdc2044471881fb741401865d7d to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMinorCompactionWithMonthlyPolicy/2be0907b8bc55af7a11c257a83fc88d6/family1/923b8bdc2044471881fb741401865d7d 2023-11-28 03:55:49,314 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithMonthlyPolicy/2be0907b8bc55af7a11c257a83fc88d6/family1/53b130e2ee5744618b0314f34850a4a9 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMinorCompactionWithMonthlyPolicy/2be0907b8bc55af7a11c257a83fc88d6/family1/53b130e2ee5744618b0314f34850a4a9 2023-11-28 03:55:49,315 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithMonthlyPolicy/2be0907b8bc55af7a11c257a83fc88d6/family1/fa0015c9744e42b09bf5e5496355a407 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMinorCompactionWithMonthlyPolicy/2be0907b8bc55af7a11c257a83fc88d6/family1/fa0015c9744e42b09bf5e5496355a407 2023-11-28 03:55:49,316 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithMonthlyPolicy/2be0907b8bc55af7a11c257a83fc88d6/family1/9a93a6dbd8b54cf4bda37a3f617da20e to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMinorCompactionWithMonthlyPolicy/2be0907b8bc55af7a11c257a83fc88d6/family1/9a93a6dbd8b54cf4bda37a3f617da20e 2023-11-28 03:55:49,318 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithMonthlyPolicy/2be0907b8bc55af7a11c257a83fc88d6/family1/e64a147830fb4588836595d056ac5345 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMinorCompactionWithMonthlyPolicy/2be0907b8bc55af7a11c257a83fc88d6/family1/e64a147830fb4588836595d056ac5345 2023-11-28 03:55:49,319 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithMonthlyPolicy/2be0907b8bc55af7a11c257a83fc88d6/family1/6d1551ff41ae4307bfb7c7ec99383e6b to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMinorCompactionWithMonthlyPolicy/2be0907b8bc55af7a11c257a83fc88d6/family1/6d1551ff41ae4307bfb7c7ec99383e6b 2023-11-28 03:55:49,321 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithMonthlyPolicy/2be0907b8bc55af7a11c257a83fc88d6/family1/d25b7f9ccb1446858757be0ec90dc462_SeqId_44_ to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMinorCompactionWithMonthlyPolicy/2be0907b8bc55af7a11c257a83fc88d6/family1/d25b7f9ccb1446858757be0ec90dc462_SeqId_44_ 2023-11-28 03:55:49,322 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/ac8573deb58847e9b4563bb4e701e73c first=Optional[r1] last=Optional[r4] 2023-11-28 03:55:49,323 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMinorCompactionWithMonthlyPolicy,,1701143738200.2be0907b8bc55af7a11c257a83fc88d6., hostname=jenkins-hbase4.apache.org,41203,1701143600653, seqNum=2 for row with hfile group [{family1,hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/ac8573deb58847e9b4563bb4e701e73c}] 2023-11-28 03:55:49,324 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41203] regionserver.HStore(763): Validating hfile at hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/ac8573deb58847e9b4563bb4e701e73c for inclusion in 2be0907b8bc55af7a11c257a83fc88d6/family1 2023-11-28 03:55:49,326 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41203] regionserver.HStore(775): HFile bounds: first=r1 last=r4 2023-11-28 03:55:49,326 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41203] regionserver.HStore(777): Region bounds: first= last= 2023-11-28 03:55:49,326 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41203] regionserver.HRegion(2446): Flush status journal for 2be0907b8bc55af7a11c257a83fc88d6: 2023-11-28 03:55:49,327 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41203] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/ac8573deb58847e9b4563bb4e701e73c to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testMinorCompactionWithMonthlyPolicy__f31r89l2gpcboqf4pkfmak2gqosu3qjsbfvc9e8i1vfdfp21u05j4f78j8rsm3fo/family1/ac8573deb58847e9b4563bb4e701e73c 2023-11-28 03:55:49,330 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41203] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testMinorCompactionWithMonthlyPolicy__f31r89l2gpcboqf4pkfmak2gqosu3qjsbfvc9e8i1vfdfp21u05j4f78j8rsm3fo/family1/ac8573deb58847e9b4563bb4e701e73c as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithMonthlyPolicy/2be0907b8bc55af7a11c257a83fc88d6/family1/70f39910e5f14158be40f0443f6733c0_SeqId_47_ 2023-11-28 03:55:49,331 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41203] regionserver.HStore(849): Loaded HFile hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testMinorCompactionWithMonthlyPolicy__f31r89l2gpcboqf4pkfmak2gqosu3qjsbfvc9e8i1vfdfp21u05j4f78j8rsm3fo/family1/ac8573deb58847e9b4563bb4e701e73c into 2be0907b8bc55af7a11c257a83fc88d6/family1 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithMonthlyPolicy/2be0907b8bc55af7a11c257a83fc88d6/family1/70f39910e5f14158be40f0443f6733c0_SeqId_47_ - updating store file list. 2023-11-28 03:55:49,335 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41203] regionserver.HStore(882): Loaded HFile hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithMonthlyPolicy/2be0907b8bc55af7a11c257a83fc88d6/family1/70f39910e5f14158be40f0443f6733c0_SeqId_47_ into 2be0907b8bc55af7a11c257a83fc88d6/family1 2023-11-28 03:55:49,335 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41203] regionserver.HStore(855): Successfully loaded hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testMinorCompactionWithMonthlyPolicy__f31r89l2gpcboqf4pkfmak2gqosu3qjsbfvc9e8i1vfdfp21u05j4f78j8rsm3fo/family1/ac8573deb58847e9b4563bb4e701e73c into 2be0907b8bc55af7a11c257a83fc88d6/family1 (new location: hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithMonthlyPolicy/2be0907b8bc55af7a11c257a83fc88d6/family1/70f39910e5f14158be40f0443f6733c0_SeqId_47_) 2023-11-28 03:55:49,335 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41203] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testMinorCompactionWithMonthlyPolicy__f31r89l2gpcboqf4pkfmak2gqosu3qjsbfvc9e8i1vfdfp21u05j4f78j8rsm3fo/family1/ac8573deb58847e9b4563bb4e701e73c 2023-11-28 03:55:49,336 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41203] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-11-28 03:55:49,337 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41203] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 182349 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-11-28 03:55:49,337 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41203] regionserver.HStore(1912): 2be0907b8bc55af7a11c257a83fc88d6/family1 is initiating minor compaction (all files) 2023-11-28 03:55:49,337 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41203] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@217e791a; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-28 03:55:49,337 INFO [RS:0;jenkins-hbase4:41203-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 2be0907b8bc55af7a11c257a83fc88d6/family1 in testMinorCompactionWithMonthlyPolicy,,1701143738200.2be0907b8bc55af7a11c257a83fc88d6. 2023-11-28 03:55:49,337 INFO [RS:0;jenkins-hbase4:41203-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithMonthlyPolicy/2be0907b8bc55af7a11c257a83fc88d6/family1/cea064f5d1454115a914f776a9041031, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithMonthlyPolicy/2be0907b8bc55af7a11c257a83fc88d6/family1/9ebe785ee7884a45ba3e3d803df703a1, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithMonthlyPolicy/2be0907b8bc55af7a11c257a83fc88d6/family1/70f39910e5f14158be40f0443f6733c0_SeqId_47_] into tmpdir=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithMonthlyPolicy/2be0907b8bc55af7a11c257a83fc88d6/.tmp, totalSize=178.1 K 2023-11-28 03:55:49,338 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] compactions.Compactor(207): Compacting cea064f5d1454115a914f776a9041031, keycount=1000, bloomtype=ROW, size=166.0 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878824053 2023-11-28 03:55:49,338 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] compactions.Compactor(207): Compacting 9ebe785ee7884a45ba3e3d803df703a1, keycount=9, bloomtype=ROW, size=6.6 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621224053 2023-11-28 03:55:49,339 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] compactions.Compactor(207): Compacting 70f39910e5f14158be40f0443f6733c0_SeqId_47_, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448965224053 2023-11-28 03:55:49,339 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151201 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-11-28 03:55:49,340 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-28 03:55:49,341 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151201e203cd16e50e4e069ac46970dab945ac to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151201e203cd16e50e4e069ac46970dab945ac 2023-11-28 03:55:49,342 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151205eca9bb999a5a47dcbecec3189c97e23a to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151205eca9bb999a5a47dcbecec3189c97e23a 2023-11-28 03:55:49,343 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201512280e934a75fd004cd3b70891f20a5abfb5 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201512280e934a75fd004cd3b70891f20a5abfb5 2023-11-28 03:55:49,345 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2015123181c17102408c40c6ae505862695a59aa to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2015123181c17102408c40c6ae505862695a59aa 2023-11-28 03:55:49,345 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMinorCompactionWithMonthlyPolicy 4 expired mob files are deleted 2023-11-28 03:55:49,345 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201' 2023-11-28 03:55:49,345 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-11-28 03:55:49,345 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-11-28 03:55:49,352 INFO [RS:0;jenkins-hbase4:41203-shortCompactions-0] throttle.PressureAwareThroughputController(145): 2be0907b8bc55af7a11c257a83fc88d6#family1#compaction#166 average throughput is 25.93 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 57.69 MB/second 2023-11-28 03:55:49,791 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithMonthlyPolicy/2be0907b8bc55af7a11c257a83fc88d6/.tmp/family1/33d2e58b797946a49c6a1f3306170c53 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithMonthlyPolicy/2be0907b8bc55af7a11c257a83fc88d6/family1/33d2e58b797946a49c6a1f3306170c53 2023-11-28 03:55:49,795 INFO [RS:0;jenkins-hbase4:41203-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 2be0907b8bc55af7a11c257a83fc88d6/family1 of 2be0907b8bc55af7a11c257a83fc88d6 into 33d2e58b797946a49c6a1f3306170c53(size=167.4 K), total size for store is 167.4 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-11-28 03:55:49,795 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 2be0907b8bc55af7a11c257a83fc88d6: 2023-11-28 03:55:49,795 INFO [RS:0;jenkins-hbase4:41203-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMinorCompactionWithMonthlyPolicy,,1701143738200.2be0907b8bc55af7a11c257a83fc88d6., storeName=family1, fileCount=3, fileSize=178.1 K (166.0 K, 6.6 K, 5.5 K), priority=2, time=1701143749337; duration=0sec 2023-11-28 03:55:49,795 DEBUG [RS:0;jenkins-hbase4:41203-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-28 03:55:49,808 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithMonthlyPolicy/2be0907b8bc55af7a11c257a83fc88d6/family1/cea064f5d1454115a914f776a9041031, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithMonthlyPolicy/2be0907b8bc55af7a11c257a83fc88d6/family1/9ebe785ee7884a45ba3e3d803df703a1, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithMonthlyPolicy/2be0907b8bc55af7a11c257a83fc88d6/family1/70f39910e5f14158be40f0443f6733c0_SeqId_47_] to archive 2023-11-28 03:55:49,808 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-28 03:55:49,810 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithMonthlyPolicy/2be0907b8bc55af7a11c257a83fc88d6/family1/cea064f5d1454115a914f776a9041031 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMinorCompactionWithMonthlyPolicy/2be0907b8bc55af7a11c257a83fc88d6/family1/cea064f5d1454115a914f776a9041031 2023-11-28 03:55:49,811 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithMonthlyPolicy/2be0907b8bc55af7a11c257a83fc88d6/family1/9ebe785ee7884a45ba3e3d803df703a1 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMinorCompactionWithMonthlyPolicy/2be0907b8bc55af7a11c257a83fc88d6/family1/9ebe785ee7884a45ba3e3d803df703a1 2023-11-28 03:55:49,812 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithMonthlyPolicy/2be0907b8bc55af7a11c257a83fc88d6/family1/70f39910e5f14158be40f0443f6733c0_SeqId_47_ to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMinorCompactionWithMonthlyPolicy/2be0907b8bc55af7a11c257a83fc88d6/family1/70f39910e5f14158be40f0443f6733c0_SeqId_47_ 2023-11-28 03:55:50,189 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151130d5f794bf00614c898aa093f5d09e5687 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151130d5f794bf00614c898aa093f5d09e5687 2023-11-28 03:55:50,189 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-11-28 03:55:50,214 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/3733be3b6b5946cdaf956f40eb5dcb32 first=Optional[r00] last=Optional[r8] 2023-11-28 03:55:50,214 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMinorCompactionWithMonthlyPolicy,,1701143738200.2be0907b8bc55af7a11c257a83fc88d6., hostname=jenkins-hbase4.apache.org,41203,1701143600653, seqNum=2 for row with hfile group [{family1,hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/3733be3b6b5946cdaf956f40eb5dcb32}] 2023-11-28 03:55:50,216 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41203] regionserver.HStore(763): Validating hfile at hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/3733be3b6b5946cdaf956f40eb5dcb32 for inclusion in 2be0907b8bc55af7a11c257a83fc88d6/family1 2023-11-28 03:55:50,219 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41203] regionserver.HStore(775): HFile bounds: first=r00 last=r8 2023-11-28 03:55:50,219 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41203] regionserver.HStore(777): Region bounds: first= last= 2023-11-28 03:55:50,219 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41203] regionserver.HRegion(2446): Flush status journal for 2be0907b8bc55af7a11c257a83fc88d6: 2023-11-28 03:55:50,219 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41203] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/3733be3b6b5946cdaf956f40eb5dcb32 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testMinorCompactionWithMonthlyPolicy__oo1fb58peu61d5jo2d4jb8eg3oqbupk2ptd3kq2o1pggdner2qjjnuv2lqalm6dd/family1/3733be3b6b5946cdaf956f40eb5dcb32 2023-11-28 03:55:50,222 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41203] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testMinorCompactionWithMonthlyPolicy__oo1fb58peu61d5jo2d4jb8eg3oqbupk2ptd3kq2o1pggdner2qjjnuv2lqalm6dd/family1/3733be3b6b5946cdaf956f40eb5dcb32 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithMonthlyPolicy/2be0907b8bc55af7a11c257a83fc88d6/family1/2b4ae2fdcef2410f9f096a545092fbad_SeqId_50_ 2023-11-28 03:55:50,222 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41203] regionserver.HStore(849): Loaded HFile hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testMinorCompactionWithMonthlyPolicy__oo1fb58peu61d5jo2d4jb8eg3oqbupk2ptd3kq2o1pggdner2qjjnuv2lqalm6dd/family1/3733be3b6b5946cdaf956f40eb5dcb32 into 2be0907b8bc55af7a11c257a83fc88d6/family1 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithMonthlyPolicy/2be0907b8bc55af7a11c257a83fc88d6/family1/2b4ae2fdcef2410f9f096a545092fbad_SeqId_50_ - updating store file list. 2023-11-28 03:55:50,226 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41203] regionserver.HStore(882): Loaded HFile hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithMonthlyPolicy/2be0907b8bc55af7a11c257a83fc88d6/family1/2b4ae2fdcef2410f9f096a545092fbad_SeqId_50_ into 2be0907b8bc55af7a11c257a83fc88d6/family1 2023-11-28 03:55:50,226 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41203] regionserver.HStore(855): Successfully loaded hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testMinorCompactionWithMonthlyPolicy__oo1fb58peu61d5jo2d4jb8eg3oqbupk2ptd3kq2o1pggdner2qjjnuv2lqalm6dd/family1/3733be3b6b5946cdaf956f40eb5dcb32 into 2be0907b8bc55af7a11c257a83fc88d6/family1 (new location: hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithMonthlyPolicy/2be0907b8bc55af7a11c257a83fc88d6/family1/2b4ae2fdcef2410f9f096a545092fbad_SeqId_50_) 2023-11-28 03:55:50,226 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41203] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testMinorCompactionWithMonthlyPolicy__oo1fb58peu61d5jo2d4jb8eg3oqbupk2ptd3kq2o1pggdner2qjjnuv2lqalm6dd/family1/3733be3b6b5946cdaf956f40eb5dcb32 2023-11-28 03:55:50,228 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-11-28 03:55:50,229 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-28 03:55:50,230 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151120f98f6bfd83834e9cb22f87abd0077929 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151120f98f6bfd83834e9cb22f87abd0077929 2023-11-28 03:55:50,231 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151128d976405866414a7c90816707e92bbf61 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151128d976405866414a7c90816707e92bbf61 2023-11-28 03:55:50,233 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2015113033df26fab23641fbaeaa982ab67fc807 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2015113033df26fab23641fbaeaa982ab67fc807 2023-11-28 03:55:50,233 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMinorCompactionWithMonthlyPolicy 3 expired mob files are deleted 2023-11-28 03:55:50,233 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101' 2023-11-28 03:55:50,233 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-11-28 03:55:50,233 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-11-28 03:55:50,233 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-11-28 03:55:50,233 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-11-28 03:55:50,233 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] mob.MobUtils(840): end MOB compaction of files for table='testMinorCompactionWithMonthlyPolicy', column='family1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-11-28 03:55:50,235 DEBUG [PEWorker-2] locking.LockProcedure(242): UNLOCKED pid=115, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-11-28 03:55:50,236 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=115, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE in 1.9250 sec 2023-11-28 03:55:50,443 INFO [Listener at localhost/42853] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMinorCompactionWithMonthlyPolicy Thread=263 (was 254) - Thread LEAK? -, OpenFileDescriptor=687 (was 673) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=204 (was 174) - SystemLoadAverage LEAK? -, ProcessCount=168 (was 168), AvailableMemoryMB=6609 (was 6620) 2023-11-28 03:55:50,455 INFO [Listener at localhost/42853] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionFromAdmin Thread=263, OpenFileDescriptor=687, MaxFileDescriptor=60000, SystemLoadAverage=204, ProcessCount=168, AvailableMemoryMB=6609 2023-11-28 03:55:50,474 DEBUG [Listener at localhost/42853] crypto.Encryption(565): Installed org.apache.hadoop.hbase.io.crypto.KeyProviderForTesting into key provider cache 2023-11-28 03:55:50,533 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testMajorCompactionFromAdmin', {NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10M\x85\x8F\xD8r\xCA\x1B\xD14\xA5[\x19\xC9j\xFDo"\x10G\x04\xCB\x1B\xB0\xAF\xAA\xE5_\xD7\xF9\x11pU\xBF\xC2*\x10\xEB\xAAf\x1C(\xFD\xD1\x9B23\xB2r\x0D\x1Di\xCD2\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} 2023-11-28 03:55:50,534 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] procedure2.ProcedureExecutor(1028): Stored pid=116, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionFromAdmin 2023-11-28 03:55:50,536 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_PRE_OPERATION 2023-11-28 03:55:50,536 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] master.MasterRpcServices(700): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionFromAdmin" procId is: 116 2023-11-28 03:55:50,536 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-11-28 03:55:50,537 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] master.MasterRpcServices(1230): Checking to see if procedure is done pid=116 2023-11-28 03:55:50,538 DEBUG [HFileArchiver-20] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/.tmp/data/default/testMajorCompactionFromAdmin/b2248d1fbdc8030e0b590971a233e4f2 2023-11-28 03:55:50,538 DEBUG [HFileArchiver-21] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/.tmp/data/default/testMajorCompactionFromAdmin/6442e6cb874dc41685a9cdfd56a2179e 2023-11-28 03:55:50,539 DEBUG [HFileArchiver-22] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/.tmp/data/default/testMajorCompactionFromAdmin/45c07d545bdbc2f92898a848086adde0 2023-11-28 03:55:50,539 DEBUG [HFileArchiver-20] backup.HFileArchiver(153): Directory hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/.tmp/data/default/testMajorCompactionFromAdmin/b2248d1fbdc8030e0b590971a233e4f2 empty. 2023-11-28 03:55:50,539 DEBUG [HFileArchiver-22] backup.HFileArchiver(153): Directory hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/.tmp/data/default/testMajorCompactionFromAdmin/45c07d545bdbc2f92898a848086adde0 empty. 2023-11-28 03:55:50,539 DEBUG [HFileArchiver-21] backup.HFileArchiver(153): Directory hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/.tmp/data/default/testMajorCompactionFromAdmin/6442e6cb874dc41685a9cdfd56a2179e empty. 2023-11-28 03:55:50,539 DEBUG [HFileArchiver-20] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/.tmp/data/default/testMajorCompactionFromAdmin/b2248d1fbdc8030e0b590971a233e4f2 2023-11-28 03:55:50,539 DEBUG [HFileArchiver-22] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/.tmp/data/default/testMajorCompactionFromAdmin/45c07d545bdbc2f92898a848086adde0 2023-11-28 03:55:50,539 DEBUG [HFileArchiver-21] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/.tmp/data/default/testMajorCompactionFromAdmin/6442e6cb874dc41685a9cdfd56a2179e 2023-11-28 03:55:50,539 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived testMajorCompactionFromAdmin regions 2023-11-28 03:55:50,638 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] master.MasterRpcServices(1230): Checking to see if procedure is done pid=116 2023-11-28 03:55:50,839 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] master.MasterRpcServices(1230): Checking to see if procedure is done pid=116 2023-11-28 03:55:50,947 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/.tmp/data/default/testMajorCompactionFromAdmin/.tabledesc/.tableinfo.0000000001 2023-11-28 03:55:50,948 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(7675): creating {ENCODED => 45c07d545bdbc2f92898a848086adde0, NAME => 'testMajorCompactionFromAdmin,2,1701143750499.45c07d545bdbc2f92898a848086adde0.', STARTKEY => '2', ENDKEY => ''}, tableDescriptor='testMajorCompactionFromAdmin', {NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10M\x85\x8F\xD8r\xCA\x1B\xD14\xA5[\x19\xC9j\xFDo"\x10G\x04\xCB\x1B\xB0\xAF\xAA\xE5_\xD7\xF9\x11pU\xBF\xC2*\x10\xEB\xAAf\x1C(\xFD\xD1\x9B23\xB2r\x0D\x1Di\xCD2\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/.tmp 2023-11-28 03:55:50,948 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(7675): creating {ENCODED => b2248d1fbdc8030e0b590971a233e4f2, NAME => 'testMajorCompactionFromAdmin,,1701143750499.b2248d1fbdc8030e0b590971a233e4f2.', STARTKEY => '', ENDKEY => '1'}, tableDescriptor='testMajorCompactionFromAdmin', {NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10M\x85\x8F\xD8r\xCA\x1B\xD14\xA5[\x19\xC9j\xFDo"\x10G\x04\xCB\x1B\xB0\xAF\xAA\xE5_\xD7\xF9\x11pU\xBF\xC2*\x10\xEB\xAAf\x1C(\xFD\xD1\x9B23\xB2r\x0D\x1Di\xCD2\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/.tmp 2023-11-28 03:55:50,948 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(7675): creating {ENCODED => 6442e6cb874dc41685a9cdfd56a2179e, NAME => 'testMajorCompactionFromAdmin,1,1701143750499.6442e6cb874dc41685a9cdfd56a2179e.', STARTKEY => '1', ENDKEY => '2'}, tableDescriptor='testMajorCompactionFromAdmin', {NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10M\x85\x8F\xD8r\xCA\x1B\xD14\xA5[\x19\xC9j\xFDo"\x10G\x04\xCB\x1B\xB0\xAF\xAA\xE5_\xD7\xF9\x11pU\xBF\xC2*\x10\xEB\xAAf\x1C(\xFD\xD1\x9B23\xB2r\x0D\x1Di\xCD2\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/.tmp 2023-11-28 03:55:51,140 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] master.MasterRpcServices(1230): Checking to see if procedure is done pid=116 2023-11-28 03:55:51,360 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(866): Instantiated testMajorCompactionFromAdmin,1,1701143750499.6442e6cb874dc41685a9cdfd56a2179e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-28 03:55:51,360 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(866): Instantiated testMajorCompactionFromAdmin,2,1701143750499.45c07d545bdbc2f92898a848086adde0.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-28 03:55:51,360 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(866): Instantiated testMajorCompactionFromAdmin,,1701143750499.b2248d1fbdc8030e0b590971a233e4f2.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-28 03:55:51,360 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1604): Closing 45c07d545bdbc2f92898a848086adde0, disabling compactions & flushes 2023-11-28 03:55:51,360 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1604): Closing 6442e6cb874dc41685a9cdfd56a2179e, disabling compactions & flushes 2023-11-28 03:55:51,360 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1626): Closing region testMajorCompactionFromAdmin,2,1701143750499.45c07d545bdbc2f92898a848086adde0. 2023-11-28 03:55:51,360 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1604): Closing b2248d1fbdc8030e0b590971a233e4f2, disabling compactions & flushes 2023-11-28 03:55:51,360 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionFromAdmin,2,1701143750499.45c07d545bdbc2f92898a848086adde0. 2023-11-28 03:55:51,360 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1626): Closing region testMajorCompactionFromAdmin,1,1701143750499.6442e6cb874dc41685a9cdfd56a2179e. 2023-11-28 03:55:51,360 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionFromAdmin,2,1701143750499.45c07d545bdbc2f92898a848086adde0. after waiting 0 ms 2023-11-28 03:55:51,361 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionFromAdmin,2,1701143750499.45c07d545bdbc2f92898a848086adde0. 2023-11-28 03:55:51,360 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1626): Closing region testMajorCompactionFromAdmin,,1701143750499.b2248d1fbdc8030e0b590971a233e4f2. 2023-11-28 03:55:51,361 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1838): Closed testMajorCompactionFromAdmin,2,1701143750499.45c07d545bdbc2f92898a848086adde0. 2023-11-28 03:55:51,361 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionFromAdmin,1,1701143750499.6442e6cb874dc41685a9cdfd56a2179e. 2023-11-28 03:55:51,361 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1558): Region close journal for 45c07d545bdbc2f92898a848086adde0: 2023-11-28 03:55:51,361 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionFromAdmin,,1701143750499.b2248d1fbdc8030e0b590971a233e4f2. 2023-11-28 03:55:51,361 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionFromAdmin,1,1701143750499.6442e6cb874dc41685a9cdfd56a2179e. after waiting 0 ms 2023-11-28 03:55:51,361 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionFromAdmin,1,1701143750499.6442e6cb874dc41685a9cdfd56a2179e. 2023-11-28 03:55:51,361 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionFromAdmin,,1701143750499.b2248d1fbdc8030e0b590971a233e4f2. after waiting 0 ms 2023-11-28 03:55:51,361 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionFromAdmin,,1701143750499.b2248d1fbdc8030e0b590971a233e4f2. 2023-11-28 03:55:51,361 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1838): Closed testMajorCompactionFromAdmin,1,1701143750499.6442e6cb874dc41685a9cdfd56a2179e. 2023-11-28 03:55:51,361 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1838): Closed testMajorCompactionFromAdmin,,1701143750499.b2248d1fbdc8030e0b590971a233e4f2. 2023-11-28 03:55:51,361 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1558): Region close journal for b2248d1fbdc8030e0b590971a233e4f2: 2023-11-28 03:55:51,361 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1558): Region close journal for 6442e6cb874dc41685a9cdfd56a2179e: 2023-11-28 03:55:51,363 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_ADD_TO_META 2023-11-28 03:55:51,364 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionFromAdmin,2,1701143750499.45c07d545bdbc2f92898a848086adde0.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1701143751364"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1701143751364"}]},"ts":"1701143751364"} 2023-11-28 03:55:51,364 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionFromAdmin,,1701143750499.b2248d1fbdc8030e0b590971a233e4f2.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1701143751364"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1701143751364"}]},"ts":"1701143751364"} 2023-11-28 03:55:51,364 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionFromAdmin,1,1701143750499.6442e6cb874dc41685a9cdfd56a2179e.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1701143751364"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1701143751364"}]},"ts":"1701143751364"} 2023-11-28 03:55:51,366 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 3 regions to meta. 2023-11-28 03:55:51,366 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-11-28 03:55:51,366 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionFromAdmin","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1701143751366"}]},"ts":"1701143751366"} 2023-11-28 03:55:51,367 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionFromAdmin, state=ENABLING in hbase:meta 2023-11-28 03:55:51,370 INFO [PEWorker-3] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=117, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=b2248d1fbdc8030e0b590971a233e4f2, ASSIGN}, {pid=118, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=6442e6cb874dc41685a9cdfd56a2179e, ASSIGN}, {pid=119, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=45c07d545bdbc2f92898a848086adde0, ASSIGN}] 2023-11-28 03:55:51,372 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=119, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=45c07d545bdbc2f92898a848086adde0, ASSIGN 2023-11-28 03:55:51,372 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=118, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=6442e6cb874dc41685a9cdfd56a2179e, ASSIGN 2023-11-28 03:55:51,372 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=117, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=b2248d1fbdc8030e0b590971a233e4f2, ASSIGN 2023-11-28 03:55:51,372 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=119, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=45c07d545bdbc2f92898a848086adde0, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,41203,1701143600653; forceNewPlan=false, retain=false 2023-11-28 03:55:51,372 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=118, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=6442e6cb874dc41685a9cdfd56a2179e, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,41203,1701143600653; forceNewPlan=false, retain=false 2023-11-28 03:55:51,372 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=117, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=b2248d1fbdc8030e0b590971a233e4f2, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,41203,1701143600653; forceNewPlan=false, retain=false 2023-11-28 03:55:51,524 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=117 updating hbase:meta row=b2248d1fbdc8030e0b590971a233e4f2, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:51,524 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=118 updating hbase:meta row=6442e6cb874dc41685a9cdfd56a2179e, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:51,524 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionFromAdmin,,1701143750499.b2248d1fbdc8030e0b590971a233e4f2.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1701143751524"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1701143751524"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1701143751524"}]},"ts":"1701143751524"} 2023-11-28 03:55:51,524 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=119 updating hbase:meta row=45c07d545bdbc2f92898a848086adde0, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:51,524 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionFromAdmin,1,1701143750499.6442e6cb874dc41685a9cdfd56a2179e.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1701143751524"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1701143751524"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1701143751524"}]},"ts":"1701143751524"} 2023-11-28 03:55:51,524 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionFromAdmin,2,1701143750499.45c07d545bdbc2f92898a848086adde0.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1701143751524"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1701143751524"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1701143751524"}]},"ts":"1701143751524"} 2023-11-28 03:55:51,525 INFO [PEWorker-3] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=120, ppid=117, state=RUNNABLE; OpenRegionProcedure b2248d1fbdc8030e0b590971a233e4f2, server=jenkins-hbase4.apache.org,41203,1701143600653}] 2023-11-28 03:55:51,526 INFO [PEWorker-2] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=121, ppid=118, state=RUNNABLE; OpenRegionProcedure 6442e6cb874dc41685a9cdfd56a2179e, server=jenkins-hbase4.apache.org,41203,1701143600653}] 2023-11-28 03:55:51,527 INFO [PEWorker-5] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=122, ppid=119, state=RUNNABLE; OpenRegionProcedure 45c07d545bdbc2f92898a848086adde0, server=jenkins-hbase4.apache.org,41203,1701143600653}] 2023-11-28 03:55:51,641 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] master.MasterRpcServices(1230): Checking to see if procedure is done pid=116 2023-11-28 03:55:51,677 DEBUG [RSProcedureDispatcher-pool-2] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:51,679 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testMajorCompactionFromAdmin,1,1701143750499.6442e6cb874dc41685a9cdfd56a2179e. 2023-11-28 03:55:51,679 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 6442e6cb874dc41685a9cdfd56a2179e, NAME => 'testMajorCompactionFromAdmin,1,1701143750499.6442e6cb874dc41685a9cdfd56a2179e.', STARTKEY => '1', ENDKEY => '2'} 2023-11-28 03:55:51,679 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionFromAdmin 6442e6cb874dc41685a9cdfd56a2179e 2023-11-28 03:55:51,680 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionFromAdmin,1,1701143750499.6442e6cb874dc41685a9cdfd56a2179e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-28 03:55:51,680 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 6442e6cb874dc41685a9cdfd56a2179e 2023-11-28 03:55:51,680 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 6442e6cb874dc41685a9cdfd56a2179e 2023-11-28 03:55:51,701 INFO [StoreOpener-6442e6cb874dc41685a9cdfd56a2179e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 6442e6cb874dc41685a9cdfd56a2179e 2023-11-28 03:55:51,702 DEBUG [StoreOpener-6442e6cb874dc41685a9cdfd56a2179e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/6442e6cb874dc41685a9cdfd56a2179e/family1 2023-11-28 03:55:51,703 DEBUG [StoreOpener-6442e6cb874dc41685a9cdfd56a2179e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/6442e6cb874dc41685a9cdfd56a2179e/family1 2023-11-28 03:55:51,703 INFO [StoreOpener-6442e6cb874dc41685a9cdfd56a2179e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 6442e6cb874dc41685a9cdfd56a2179e columnFamilyName family1 2023-11-28 03:55:51,704 INFO [StoreOpener-6442e6cb874dc41685a9cdfd56a2179e-1] regionserver.HStore(310): Store=6442e6cb874dc41685a9cdfd56a2179e/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-28 03:55:51,704 INFO [StoreOpener-6442e6cb874dc41685a9cdfd56a2179e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 6442e6cb874dc41685a9cdfd56a2179e 2023-11-28 03:55:51,706 DEBUG [StoreOpener-6442e6cb874dc41685a9cdfd56a2179e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/6442e6cb874dc41685a9cdfd56a2179e/family2 2023-11-28 03:55:51,706 DEBUG [StoreOpener-6442e6cb874dc41685a9cdfd56a2179e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/6442e6cb874dc41685a9cdfd56a2179e/family2 2023-11-28 03:55:51,706 INFO [StoreOpener-6442e6cb874dc41685a9cdfd56a2179e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 6442e6cb874dc41685a9cdfd56a2179e columnFamilyName family2 2023-11-28 03:55:51,707 INFO [StoreOpener-6442e6cb874dc41685a9cdfd56a2179e-1] regionserver.HStore(310): Store=6442e6cb874dc41685a9cdfd56a2179e/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-28 03:55:51,708 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/6442e6cb874dc41685a9cdfd56a2179e 2023-11-28 03:55:51,708 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/6442e6cb874dc41685a9cdfd56a2179e 2023-11-28 03:55:51,710 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testMajorCompactionFromAdmin descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-11-28 03:55:51,711 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 6442e6cb874dc41685a9cdfd56a2179e 2023-11-28 03:55:51,713 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/6442e6cb874dc41685a9cdfd56a2179e/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-11-28 03:55:51,713 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 6442e6cb874dc41685a9cdfd56a2179e; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60131660, jitterRate=-0.10396844148635864}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-11-28 03:55:51,713 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 6442e6cb874dc41685a9cdfd56a2179e: 2023-11-28 03:55:51,714 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2339): Post open deploy tasks for testMajorCompactionFromAdmin,1,1701143750499.6442e6cb874dc41685a9cdfd56a2179e., pid=121, masterSystemTime=1701143751677 2023-11-28 03:55:51,715 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2366): Finished post open deploy task for testMajorCompactionFromAdmin,1,1701143750499.6442e6cb874dc41685a9cdfd56a2179e. 2023-11-28 03:55:51,715 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testMajorCompactionFromAdmin,1,1701143750499.6442e6cb874dc41685a9cdfd56a2179e. 2023-11-28 03:55:51,715 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testMajorCompactionFromAdmin,2,1701143750499.45c07d545bdbc2f92898a848086adde0. 2023-11-28 03:55:51,715 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 45c07d545bdbc2f92898a848086adde0, NAME => 'testMajorCompactionFromAdmin,2,1701143750499.45c07d545bdbc2f92898a848086adde0.', STARTKEY => '2', ENDKEY => ''} 2023-11-28 03:55:51,715 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionFromAdmin 45c07d545bdbc2f92898a848086adde0 2023-11-28 03:55:51,715 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=118 updating hbase:meta row=6442e6cb874dc41685a9cdfd56a2179e, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:51,716 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionFromAdmin,2,1701143750499.45c07d545bdbc2f92898a848086adde0.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-28 03:55:51,716 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 45c07d545bdbc2f92898a848086adde0 2023-11-28 03:55:51,716 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionFromAdmin,1,1701143750499.6442e6cb874dc41685a9cdfd56a2179e.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1701143751715"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1701143751715"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1701143751715"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1701143751715"}]},"ts":"1701143751715"} 2023-11-28 03:55:51,716 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 45c07d545bdbc2f92898a848086adde0 2023-11-28 03:55:51,719 INFO [PEWorker-1] procedure2.ProcedureExecutor(1825): Finished subprocedure pid=121, resume processing ppid=118 2023-11-28 03:55:51,719 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=121, ppid=118, state=SUCCESS; OpenRegionProcedure 6442e6cb874dc41685a9cdfd56a2179e, server=jenkins-hbase4.apache.org,41203,1701143600653 in 191 msec 2023-11-28 03:55:51,720 INFO [PEWorker-4] procedure2.ProcedureExecutor(1409): Finished pid=118, ppid=116, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=6442e6cb874dc41685a9cdfd56a2179e, ASSIGN in 349 msec 2023-11-28 03:55:51,739 INFO [StoreOpener-45c07d545bdbc2f92898a848086adde0-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 45c07d545bdbc2f92898a848086adde0 2023-11-28 03:55:51,740 DEBUG [StoreOpener-45c07d545bdbc2f92898a848086adde0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/45c07d545bdbc2f92898a848086adde0/family1 2023-11-28 03:55:51,741 DEBUG [StoreOpener-45c07d545bdbc2f92898a848086adde0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/45c07d545bdbc2f92898a848086adde0/family1 2023-11-28 03:55:51,741 INFO [StoreOpener-45c07d545bdbc2f92898a848086adde0-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 45c07d545bdbc2f92898a848086adde0 columnFamilyName family1 2023-11-28 03:55:51,742 INFO [StoreOpener-45c07d545bdbc2f92898a848086adde0-1] regionserver.HStore(310): Store=45c07d545bdbc2f92898a848086adde0/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-28 03:55:51,742 INFO [StoreOpener-45c07d545bdbc2f92898a848086adde0-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 45c07d545bdbc2f92898a848086adde0 2023-11-28 03:55:51,744 DEBUG [StoreOpener-45c07d545bdbc2f92898a848086adde0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/45c07d545bdbc2f92898a848086adde0/family2 2023-11-28 03:55:51,744 DEBUG [StoreOpener-45c07d545bdbc2f92898a848086adde0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/45c07d545bdbc2f92898a848086adde0/family2 2023-11-28 03:55:51,744 INFO [StoreOpener-45c07d545bdbc2f92898a848086adde0-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 45c07d545bdbc2f92898a848086adde0 columnFamilyName family2 2023-11-28 03:55:51,745 INFO [StoreOpener-45c07d545bdbc2f92898a848086adde0-1] regionserver.HStore(310): Store=45c07d545bdbc2f92898a848086adde0/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-28 03:55:51,745 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/45c07d545bdbc2f92898a848086adde0 2023-11-28 03:55:51,746 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/45c07d545bdbc2f92898a848086adde0 2023-11-28 03:55:51,747 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testMajorCompactionFromAdmin descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-11-28 03:55:51,747 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 45c07d545bdbc2f92898a848086adde0 2023-11-28 03:55:51,748 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/45c07d545bdbc2f92898a848086adde0/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-11-28 03:55:51,749 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 45c07d545bdbc2f92898a848086adde0; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74501343, jitterRate=0.11015652120113373}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-11-28 03:55:51,749 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 45c07d545bdbc2f92898a848086adde0: 2023-11-28 03:55:51,749 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2339): Post open deploy tasks for testMajorCompactionFromAdmin,2,1701143750499.45c07d545bdbc2f92898a848086adde0., pid=122, masterSystemTime=1701143751677 2023-11-28 03:55:51,750 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2366): Finished post open deploy task for testMajorCompactionFromAdmin,2,1701143750499.45c07d545bdbc2f92898a848086adde0. 2023-11-28 03:55:51,750 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testMajorCompactionFromAdmin,2,1701143750499.45c07d545bdbc2f92898a848086adde0. 2023-11-28 03:55:51,750 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testMajorCompactionFromAdmin,,1701143750499.b2248d1fbdc8030e0b590971a233e4f2. 2023-11-28 03:55:51,751 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => b2248d1fbdc8030e0b590971a233e4f2, NAME => 'testMajorCompactionFromAdmin,,1701143750499.b2248d1fbdc8030e0b590971a233e4f2.', STARTKEY => '', ENDKEY => '1'} 2023-11-28 03:55:51,751 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=119 updating hbase:meta row=45c07d545bdbc2f92898a848086adde0, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:51,751 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionFromAdmin b2248d1fbdc8030e0b590971a233e4f2 2023-11-28 03:55:51,751 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionFromAdmin,2,1701143750499.45c07d545bdbc2f92898a848086adde0.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1701143751751"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1701143751751"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1701143751751"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1701143751751"}]},"ts":"1701143751751"} 2023-11-28 03:55:51,751 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionFromAdmin,,1701143750499.b2248d1fbdc8030e0b590971a233e4f2.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-28 03:55:51,751 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for b2248d1fbdc8030e0b590971a233e4f2 2023-11-28 03:55:51,751 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for b2248d1fbdc8030e0b590971a233e4f2 2023-11-28 03:55:51,753 INFO [PEWorker-5] procedure2.ProcedureExecutor(1825): Finished subprocedure pid=122, resume processing ppid=119 2023-11-28 03:55:51,753 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=122, ppid=119, state=SUCCESS; OpenRegionProcedure 45c07d545bdbc2f92898a848086adde0, server=jenkins-hbase4.apache.org,41203,1701143600653 in 226 msec 2023-11-28 03:55:51,753 INFO [PEWorker-3] procedure2.ProcedureExecutor(1409): Finished pid=119, ppid=116, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=45c07d545bdbc2f92898a848086adde0, ASSIGN in 383 msec 2023-11-28 03:55:51,781 INFO [StoreOpener-b2248d1fbdc8030e0b590971a233e4f2-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region b2248d1fbdc8030e0b590971a233e4f2 2023-11-28 03:55:51,782 DEBUG [StoreOpener-b2248d1fbdc8030e0b590971a233e4f2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/b2248d1fbdc8030e0b590971a233e4f2/family1 2023-11-28 03:55:51,782 DEBUG [StoreOpener-b2248d1fbdc8030e0b590971a233e4f2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/b2248d1fbdc8030e0b590971a233e4f2/family1 2023-11-28 03:55:51,782 INFO [StoreOpener-b2248d1fbdc8030e0b590971a233e4f2-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region b2248d1fbdc8030e0b590971a233e4f2 columnFamilyName family1 2023-11-28 03:55:51,783 INFO [StoreOpener-b2248d1fbdc8030e0b590971a233e4f2-1] regionserver.HStore(310): Store=b2248d1fbdc8030e0b590971a233e4f2/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-28 03:55:51,783 INFO [StoreOpener-b2248d1fbdc8030e0b590971a233e4f2-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region b2248d1fbdc8030e0b590971a233e4f2 2023-11-28 03:55:51,784 DEBUG [StoreOpener-b2248d1fbdc8030e0b590971a233e4f2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/b2248d1fbdc8030e0b590971a233e4f2/family2 2023-11-28 03:55:51,784 DEBUG [StoreOpener-b2248d1fbdc8030e0b590971a233e4f2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/b2248d1fbdc8030e0b590971a233e4f2/family2 2023-11-28 03:55:51,785 INFO [StoreOpener-b2248d1fbdc8030e0b590971a233e4f2-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region b2248d1fbdc8030e0b590971a233e4f2 columnFamilyName family2 2023-11-28 03:55:51,785 INFO [StoreOpener-b2248d1fbdc8030e0b590971a233e4f2-1] regionserver.HStore(310): Store=b2248d1fbdc8030e0b590971a233e4f2/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-28 03:55:51,786 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/b2248d1fbdc8030e0b590971a233e4f2 2023-11-28 03:55:51,786 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/b2248d1fbdc8030e0b590971a233e4f2 2023-11-28 03:55:51,787 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testMajorCompactionFromAdmin descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-11-28 03:55:51,788 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for b2248d1fbdc8030e0b590971a233e4f2 2023-11-28 03:55:51,789 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/b2248d1fbdc8030e0b590971a233e4f2/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-11-28 03:55:51,789 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened b2248d1fbdc8030e0b590971a233e4f2; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=66901483, jitterRate=-0.003090217709541321}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-11-28 03:55:51,789 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for b2248d1fbdc8030e0b590971a233e4f2: 2023-11-28 03:55:51,790 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2339): Post open deploy tasks for testMajorCompactionFromAdmin,,1701143750499.b2248d1fbdc8030e0b590971a233e4f2., pid=120, masterSystemTime=1701143751677 2023-11-28 03:55:51,791 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2366): Finished post open deploy task for testMajorCompactionFromAdmin,,1701143750499.b2248d1fbdc8030e0b590971a233e4f2. 2023-11-28 03:55:51,791 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testMajorCompactionFromAdmin,,1701143750499.b2248d1fbdc8030e0b590971a233e4f2. 2023-11-28 03:55:51,791 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=117 updating hbase:meta row=b2248d1fbdc8030e0b590971a233e4f2, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:51,792 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionFromAdmin,,1701143750499.b2248d1fbdc8030e0b590971a233e4f2.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1701143751791"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1701143751791"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1701143751791"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1701143751791"}]},"ts":"1701143751791"} 2023-11-28 03:55:51,794 INFO [PEWorker-2] procedure2.ProcedureExecutor(1825): Finished subprocedure pid=120, resume processing ppid=117 2023-11-28 03:55:51,794 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=120, ppid=117, state=SUCCESS; OpenRegionProcedure b2248d1fbdc8030e0b590971a233e4f2, server=jenkins-hbase4.apache.org,41203,1701143600653 in 268 msec 2023-11-28 03:55:51,795 INFO [PEWorker-1] procedure2.ProcedureExecutor(1825): Finished subprocedure pid=117, resume processing ppid=116 2023-11-28 03:55:51,795 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=117, ppid=116, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=b2248d1fbdc8030e0b590971a233e4f2, ASSIGN in 424 msec 2023-11-28 03:55:51,795 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-11-28 03:55:51,795 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionFromAdmin","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1701143751795"}]},"ts":"1701143751795"} 2023-11-28 03:55:51,796 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionFromAdmin, state=ENABLED in hbase:meta 2023-11-28 03:55:51,798 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_POST_OPERATION 2023-11-28 03:55:51,799 INFO [PEWorker-4] procedure2.ProcedureExecutor(1409): Finished pid=116, state=SUCCESS; CreateTableProcedure table=testMajorCompactionFromAdmin in 1.2650 sec 2023-11-28 03:55:52,398 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionFromAdmin' 2023-11-28 03:55:52,641 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] master.MasterRpcServices(1230): Checking to see if procedure is done pid=116 2023-11-28 03:55:52,642 INFO [Listener at localhost/42853] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionFromAdmin, procId: 116 completed 2023-11-28 03:55:52,647 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41203] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,,1701143750499.b2248d1fbdc8030e0b590971a233e4f2. with WAL disabled. Data may be lost in the event of a crash. 2023-11-28 03:55:52,647 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41203] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,1,1701143750499.6442e6cb874dc41685a9cdfd56a2179e. with WAL disabled. Data may be lost in the event of a crash. 2023-11-28 03:55:52,647 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41203] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,2,1701143750499.45c07d545bdbc2f92898a848086adde0. with WAL disabled. Data may be lost in the event of a crash. 2023-11-28 03:55:52,649 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-28 03:55:52,650 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionFromAdmin 2023-11-28 03:55:52,650 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionFromAdmin' 2023-11-28 03:55:52,650 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:55:52,650 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionFromAdmin' starting 'acquire' 2023-11-28 03:55:52,650 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionFromAdmin', kicking off acquire phase on members. 2023-11-28 03:55:52,651 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-28 03:55:52,651 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-11-28 03:55:52,653 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:52,653 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:55:52,653 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:55:52,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:55:52,653 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:52,653 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-28 03:55:52,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-11-28 03:55:52,654 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-28 03:55:52,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-28 03:55:52,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-11-28 03:55:52,654 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionFromAdmin 2023-11-28 03:55:52,654 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionFromAdmin 2023-11-28 03:55:52,654 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionFromAdmin' with timeout 60000ms 2023-11-28 03:55:52,654 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:55:52,654 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionFromAdmin' starting 'acquire' stage 2023-11-28 03:55:52,655 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,,1701143750499.b2248d1fbdc8030e0b590971a233e4f2. 2023-11-28 03:55:52,655 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,2,1701143750499.45c07d545bdbc2f92898a848086adde0. 2023-11-28 03:55:52,655 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,,1701143750499.b2248d1fbdc8030e0b590971a233e4f2. started... 2023-11-28 03:55:52,655 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,2,1701143750499.45c07d545bdbc2f92898a848086adde0. started... 2023-11-28 03:55:52,655 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-11-28 03:55:52,655 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-28 03:55:52,655 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,1,1701143750499.6442e6cb874dc41685a9cdfd56a2179e. 2023-11-28 03:55:52,655 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing b2248d1fbdc8030e0b590971a233e4f2 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-11-28 03:55:52,655 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 45c07d545bdbc2f92898a848086adde0 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-11-28 03:55:52,655 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,1,1701143750499.6442e6cb874dc41685a9cdfd56a2179e. started... 2023-11-28 03:55:52,656 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 6442e6cb874dc41685a9cdfd56a2179e 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-11-28 03:55:52,703 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20231128ca05136ff25842999569be376219b42b to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20231128ca05136ff25842999569be376219b42b 2023-11-28 03:55:52,716 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/45c07d545bdbc2f92898a848086adde0/.tmp/family1/32df831f9edc48e0a26c4c619f13ce3b 2023-11-28 03:55:52,749 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202311282a597e5f42cb4c68a3f435e3fc16b17d to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c81e728d9d4c2f636f067f89cc14862c202311282a597e5f42cb4c68a3f435e3fc16b17d 2023-11-28 03:55:53,102 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023112811bd7bd295fb4b25b693d333431cf856 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b2023112811bd7bd295fb4b25b693d333431cf856 2023-11-28 03:55:53,102 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202311284bef538b89cc416ba999c2eb9450c187 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202311284bef538b89cc416ba999c2eb9450c187 2023-11-28 03:55:53,153 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/45c07d545bdbc2f92898a848086adde0/.tmp/family2/52498b831e88442095d4a1cccc76b337 2023-11-28 03:55:53,157 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/45c07d545bdbc2f92898a848086adde0/.tmp/family1/32df831f9edc48e0a26c4c619f13ce3b as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/45c07d545bdbc2f92898a848086adde0/family1/32df831f9edc48e0a26c4c619f13ce3b 2023-11-28 03:55:53,181 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/45c07d545bdbc2f92898a848086adde0/family1/32df831f9edc48e0a26c4c619f13ce3b, entries=4, sequenceid=5, filesize=5.5 K 2023-11-28 03:55:53,182 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/45c07d545bdbc2f92898a848086adde0/.tmp/family2/52498b831e88442095d4a1cccc76b337 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/45c07d545bdbc2f92898a848086adde0/family2/52498b831e88442095d4a1cccc76b337 2023-11-28 03:55:53,185 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/45c07d545bdbc2f92898a848086adde0/family2/52498b831e88442095d4a1cccc76b337, entries=2, sequenceid=5, filesize=5.1 K 2023-11-28 03:55:53,185 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 45c07d545bdbc2f92898a848086adde0 in 530ms, sequenceid=5, compaction requested=false 2023-11-28 03:55:53,186 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 45c07d545bdbc2f92898a848086adde0: 2023-11-28 03:55:53,186 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,2,1701143750499.45c07d545bdbc2f92898a848086adde0. 2023-11-28 03:55:53,186 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-11-28 03:55:53,524 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/6442e6cb874dc41685a9cdfd56a2179e/.tmp/family1/3c668f19e1ce4476b89ff27928db8028 2023-11-28 03:55:53,525 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/b2248d1fbdc8030e0b590971a233e4f2/.tmp/family1/c72b173385884f6e8fefb324934c1840 2023-11-28 03:55:53,966 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20231128825e82f92f214b229969c1b55d7a8832 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/d41d8cd98f00b204e9800998ecf8427e20231128825e82f92f214b229969c1b55d7a8832 2023-11-28 03:55:53,967 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202311289c7e8aa5c67c4da5a682a18f30299979 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c4ca4238a0b923820dcc509a6f75849b202311289c7e8aa5c67c4da5a682a18f30299979 2023-11-28 03:55:54,371 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/b2248d1fbdc8030e0b590971a233e4f2/.tmp/family2/8df51ba4358b460b86db02295d3ef6d8 2023-11-28 03:55:54,374 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/6442e6cb874dc41685a9cdfd56a2179e/.tmp/family2/6352c64aff6a4e36bc7588737f37a8bd 2023-11-28 03:55:54,379 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/b2248d1fbdc8030e0b590971a233e4f2/.tmp/family1/c72b173385884f6e8fefb324934c1840 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/b2248d1fbdc8030e0b590971a233e4f2/family1/c72b173385884f6e8fefb324934c1840 2023-11-28 03:55:54,381 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/6442e6cb874dc41685a9cdfd56a2179e/.tmp/family1/3c668f19e1ce4476b89ff27928db8028 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/6442e6cb874dc41685a9cdfd56a2179e/family1/3c668f19e1ce4476b89ff27928db8028 2023-11-28 03:55:54,413 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/b2248d1fbdc8030e0b590971a233e4f2/family1/c72b173385884f6e8fefb324934c1840, entries=4, sequenceid=5, filesize=5.5 K 2023-11-28 03:55:54,414 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/b2248d1fbdc8030e0b590971a233e4f2/.tmp/family2/8df51ba4358b460b86db02295d3ef6d8 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/b2248d1fbdc8030e0b590971a233e4f2/family2/8df51ba4358b460b86db02295d3ef6d8 2023-11-28 03:55:54,429 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/6442e6cb874dc41685a9cdfd56a2179e/family1/3c668f19e1ce4476b89ff27928db8028, entries=4, sequenceid=5, filesize=5.5 K 2023-11-28 03:55:54,430 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/6442e6cb874dc41685a9cdfd56a2179e/.tmp/family2/6352c64aff6a4e36bc7588737f37a8bd as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/6442e6cb874dc41685a9cdfd56a2179e/family2/6352c64aff6a4e36bc7588737f37a8bd 2023-11-28 03:55:54,430 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/b2248d1fbdc8030e0b590971a233e4f2/family2/8df51ba4358b460b86db02295d3ef6d8, entries=2, sequenceid=5, filesize=5.1 K 2023-11-28 03:55:54,431 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for b2248d1fbdc8030e0b590971a233e4f2 in 1776ms, sequenceid=5, compaction requested=false 2023-11-28 03:55:54,431 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for b2248d1fbdc8030e0b590971a233e4f2: 2023-11-28 03:55:54,431 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,,1701143750499.b2248d1fbdc8030e0b590971a233e4f2. 2023-11-28 03:55:54,431 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-11-28 03:55:54,436 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/6442e6cb874dc41685a9cdfd56a2179e/family2/6352c64aff6a4e36bc7588737f37a8bd, entries=2, sequenceid=5, filesize=5.1 K 2023-11-28 03:55:54,437 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 6442e6cb874dc41685a9cdfd56a2179e in 1781ms, sequenceid=5, compaction requested=false 2023-11-28 03:55:54,437 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 6442e6cb874dc41685a9cdfd56a2179e: 2023-11-28 03:55:54,437 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,1,1701143750499.6442e6cb874dc41685a9cdfd56a2179e. 2023-11-28 03:55:54,437 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-11-28 03:55:54,437 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-11-28 03:55:54,437 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:54,437 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionFromAdmin' locally acquired 2023-11-28 03:55:54,437 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure (testMajorCompactionFromAdmin) in zk 2023-11-28 03:55:54,439 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-28 03:55:54,439 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:54,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:54,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:55:54,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:55:54,439 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-28 03:55:54,439 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionFromAdmin' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-28 03:55:54,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:55:54,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:55:54,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-11-28 03:55:54,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:54,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:55:54,441 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure 'testMajorCompactionFromAdmin' on coordinator 2023-11-28 03:55:54,441 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7ccd9ec2[Count = 0] remaining members to acquire global barrier 2023-11-28 03:55:54,441 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionFromAdmin' starting 'in-barrier' execution. 2023-11-28 03:55:54,442 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-28 03:55:54,443 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-28 03:55:54,443 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-28 03:55:54,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-28 03:55:54,443 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionFromAdmin' received 'reached' from coordinator. 2023-11-28 03:55:54,443 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionFromAdmin' locally completed 2023-11-28 03:55:54,443 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionFromAdmin' completed for member 'jenkins-hbase4.apache.org,41203,1701143600653' in zk 2023-11-28 03:55:54,443 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:54,443 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-28 03:55:54,444 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionFromAdmin' has notified controller of completion 2023-11-28 03:55:54,444 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:55:54,444 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:54,444 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionFromAdmin' completed. 2023-11-28 03:55:54,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:54,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:55:54,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:55:54,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:55:54,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:55:54,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-11-28 03:55:54,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:54,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:55:54,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-11-28 03:55:54,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:54,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionFromAdmin' member 'jenkins-hbase4.apache.org,41203,1701143600653': 2023-11-28 03:55:54,448 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' released barrier for procedure'testMajorCompactionFromAdmin', counting down latch. Waiting for 0 more 2023-11-28 03:55:54,448 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionFromAdmin' execution completed 2023-11-28 03:55:54,448 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-28 03:55:54,449 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-28 03:55:54,449 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionFromAdmin 2023-11-28 03:55:54,449 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionFromAdminincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-28 03:55:54,455 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-28 03:55:54,455 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-28 03:55:54,455 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:55:54,455 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-28 03:55:54,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-28 03:55:54,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:55:54,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:55:54,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-28 03:55:54,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:55:54,456 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:55:54,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:55:54,457 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:54,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-11-28 03:55:54,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-28 03:55:54,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:55:54,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-11-28 03:55:54,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:54,459 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:54,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:55:54,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-11-28 03:55:54,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:54,461 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:54,462 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-11-28 03:55:54,462 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-11-28 03:55:54,462 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:54,462 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-28 03:55:54,462 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-28 03:55:54,462 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:55:54,461 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:55:54,462 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:55:54,462 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:55:54,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:55:54,462 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(399): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin because node does not exist (not an error) 2023-11-28 03:55:54,462 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionFromAdmin' 2023-11-28 03:55:54,462 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-28 03:55:54,462 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-28 03:55:54,462 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionFromAdmin'' to complete. (max 300000 ms per retry) 2023-11-28 03:55:54,463 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-28 03:55:54,462 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:55:54,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:55:54,563 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-28 03:55:54,564 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-28 03:55:54,565 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41203] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,,1701143750499.b2248d1fbdc8030e0b590971a233e4f2. with WAL disabled. Data may be lost in the event of a crash. 2023-11-28 03:55:54,565 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41203] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,1,1701143750499.6442e6cb874dc41685a9cdfd56a2179e. with WAL disabled. Data may be lost in the event of a crash. 2023-11-28 03:55:54,565 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41203] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,2,1701143750499.45c07d545bdbc2f92898a848086adde0. with WAL disabled. Data may be lost in the event of a crash. 2023-11-28 03:55:54,568 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-28 03:55:54,570 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionFromAdmin was in running list but was completed. Accepting new attempt. 2023-11-28 03:55:54,570 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionFromAdmin 2023-11-28 03:55:54,570 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionFromAdmin' 2023-11-28 03:55:54,570 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:55:54,570 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionFromAdmin' starting 'acquire' 2023-11-28 03:55:54,570 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionFromAdmin', kicking off acquire phase on members. 2023-11-28 03:55:54,570 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-28 03:55:54,570 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-11-28 03:55:54,572 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:54,572 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:55:54,572 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:55:54,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:55:54,572 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:54,572 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-28 03:55:54,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-11-28 03:55:54,573 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-28 03:55:54,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-28 03:55:54,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-11-28 03:55:54,573 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionFromAdmin 2023-11-28 03:55:54,574 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionFromAdmin is still present, removing 2023-11-28 03:55:54,574 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionFromAdmin 2023-11-28 03:55:54,574 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionFromAdmin' with timeout 60000ms 2023-11-28 03:55:54,574 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:55:54,574 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionFromAdmin' starting 'acquire' stage 2023-11-28 03:55:54,574 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,,1701143750499.b2248d1fbdc8030e0b590971a233e4f2. 2023-11-28 03:55:54,574 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,,1701143750499.b2248d1fbdc8030e0b590971a233e4f2. started... 2023-11-28 03:55:54,574 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,2,1701143750499.45c07d545bdbc2f92898a848086adde0. 2023-11-28 03:55:54,574 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-11-28 03:55:54,574 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,1,1701143750499.6442e6cb874dc41685a9cdfd56a2179e. 2023-11-28 03:55:54,575 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,1,1701143750499.6442e6cb874dc41685a9cdfd56a2179e. started... 2023-11-28 03:55:54,574 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,2,1701143750499.45c07d545bdbc2f92898a848086adde0. started... 2023-11-28 03:55:54,574 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing b2248d1fbdc8030e0b590971a233e4f2 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-11-28 03:55:54,574 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-28 03:55:54,575 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 45c07d545bdbc2f92898a848086adde0 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-11-28 03:55:54,575 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 6442e6cb874dc41685a9cdfd56a2179e 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-11-28 03:55:54,637 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023112892c014b50df043d6ab29ac6d6c8f5436 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b2023112892c014b50df043d6ab29ac6d6c8f5436 2023-11-28 03:55:54,637 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20231128c1a0e78578c04930ad0ab004ec4b7c02 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20231128c1a0e78578c04930ad0ab004ec4b7c02 2023-11-28 03:55:54,638 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20231128f66c6719519a489994bca7c8c780000d to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20231128f66c6719519a489994bca7c8c780000d 2023-11-28 03:55:54,665 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/6442e6cb874dc41685a9cdfd56a2179e/.tmp/family1/6dd6971d94d64d53a807de63c42e191f 2023-11-28 03:55:54,724 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20231128b0de6cb1e3a64481b5568e6133e35106 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c4ca4238a0b923820dcc509a6f75849b20231128b0de6cb1e3a64481b5568e6133e35106 2023-11-28 03:55:55,055 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/45c07d545bdbc2f92898a848086adde0/.tmp/family1/421a2ed221404530a8265481bd5e5ad8 2023-11-28 03:55:55,059 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/b2248d1fbdc8030e0b590971a233e4f2/.tmp/family1/d4a3e5e991fa43e39a41e6dfc30b3de9 2023-11-28 03:55:55,088 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20231128a8a9897f803449738d4c8a44aa046e76 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/d41d8cd98f00b204e9800998ecf8427e20231128a8a9897f803449738d4c8a44aa046e76 2023-11-28 03:55:55,092 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/b2248d1fbdc8030e0b590971a233e4f2/.tmp/family2/29407608efa54d139b093bd6a751812a 2023-11-28 03:55:55,095 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/b2248d1fbdc8030e0b590971a233e4f2/.tmp/family1/d4a3e5e991fa43e39a41e6dfc30b3de9 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/b2248d1fbdc8030e0b590971a233e4f2/family1/d4a3e5e991fa43e39a41e6dfc30b3de9 2023-11-28 03:55:55,120 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/b2248d1fbdc8030e0b590971a233e4f2/family1/d4a3e5e991fa43e39a41e6dfc30b3de9, entries=4, sequenceid=9, filesize=5.5 K 2023-11-28 03:55:55,120 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/b2248d1fbdc8030e0b590971a233e4f2/.tmp/family2/29407608efa54d139b093bd6a751812a as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/b2248d1fbdc8030e0b590971a233e4f2/family2/29407608efa54d139b093bd6a751812a 2023-11-28 03:55:55,123 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/b2248d1fbdc8030e0b590971a233e4f2/family2/29407608efa54d139b093bd6a751812a, entries=2, sequenceid=9, filesize=5.1 K 2023-11-28 03:55:55,124 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for b2248d1fbdc8030e0b590971a233e4f2 in 550ms, sequenceid=9, compaction requested=false 2023-11-28 03:55:55,124 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for b2248d1fbdc8030e0b590971a233e4f2: 2023-11-28 03:55:55,124 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,,1701143750499.b2248d1fbdc8030e0b590971a233e4f2. 2023-11-28 03:55:55,124 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-11-28 03:55:55,134 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/6442e6cb874dc41685a9cdfd56a2179e/.tmp/family2/ac838129541241b7bd8bf8d34e279b81 2023-11-28 03:55:55,137 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/6442e6cb874dc41685a9cdfd56a2179e/.tmp/family1/6dd6971d94d64d53a807de63c42e191f as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/6442e6cb874dc41685a9cdfd56a2179e/family1/6dd6971d94d64d53a807de63c42e191f 2023-11-28 03:55:55,160 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/6442e6cb874dc41685a9cdfd56a2179e/family1/6dd6971d94d64d53a807de63c42e191f, entries=4, sequenceid=9, filesize=5.5 K 2023-11-28 03:55:55,161 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/6442e6cb874dc41685a9cdfd56a2179e/.tmp/family2/ac838129541241b7bd8bf8d34e279b81 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/6442e6cb874dc41685a9cdfd56a2179e/family2/ac838129541241b7bd8bf8d34e279b81 2023-11-28 03:55:55,163 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/6442e6cb874dc41685a9cdfd56a2179e/family2/ac838129541241b7bd8bf8d34e279b81, entries=2, sequenceid=9, filesize=5.1 K 2023-11-28 03:55:55,164 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 6442e6cb874dc41685a9cdfd56a2179e in 589ms, sequenceid=9, compaction requested=false 2023-11-28 03:55:55,164 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 6442e6cb874dc41685a9cdfd56a2179e: 2023-11-28 03:55:55,164 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,1,1701143750499.6442e6cb874dc41685a9cdfd56a2179e. 2023-11-28 03:55:55,164 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-11-28 03:55:55,488 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023112824685ac810094483be1a8c743ae5bc30 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c81e728d9d4c2f636f067f89cc14862c2023112824685ac810094483be1a8c743ae5bc30 2023-11-28 03:55:55,493 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/45c07d545bdbc2f92898a848086adde0/.tmp/family2/c8f31136ec6c48f58e1c448f21f608fc 2023-11-28 03:55:55,497 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/45c07d545bdbc2f92898a848086adde0/.tmp/family1/421a2ed221404530a8265481bd5e5ad8 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/45c07d545bdbc2f92898a848086adde0/family1/421a2ed221404530a8265481bd5e5ad8 2023-11-28 03:55:55,534 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/45c07d545bdbc2f92898a848086adde0/family1/421a2ed221404530a8265481bd5e5ad8, entries=4, sequenceid=9, filesize=5.5 K 2023-11-28 03:55:55,535 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/45c07d545bdbc2f92898a848086adde0/.tmp/family2/c8f31136ec6c48f58e1c448f21f608fc as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/45c07d545bdbc2f92898a848086adde0/family2/c8f31136ec6c48f58e1c448f21f608fc 2023-11-28 03:55:55,539 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/45c07d545bdbc2f92898a848086adde0/family2/c8f31136ec6c48f58e1c448f21f608fc, entries=2, sequenceid=9, filesize=5.1 K 2023-11-28 03:55:55,540 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 45c07d545bdbc2f92898a848086adde0 in 965ms, sequenceid=9, compaction requested=false 2023-11-28 03:55:55,540 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 45c07d545bdbc2f92898a848086adde0: 2023-11-28 03:55:55,540 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,2,1701143750499.45c07d545bdbc2f92898a848086adde0. 2023-11-28 03:55:55,540 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-11-28 03:55:55,540 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-11-28 03:55:55,540 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:55,540 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionFromAdmin' locally acquired 2023-11-28 03:55:55,540 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure (testMajorCompactionFromAdmin) in zk 2023-11-28 03:55:55,542 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-28 03:55:55,543 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:55,543 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-28 03:55:55,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:55,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:55:55,543 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionFromAdmin' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-28 03:55:55,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:55:55,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:55:55,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:55:55,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-11-28 03:55:55,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:55,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:55:55,546 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure 'testMajorCompactionFromAdmin' on coordinator 2023-11-28 03:55:55,546 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@42a006d4[Count = 0] remaining members to acquire global barrier 2023-11-28 03:55:55,546 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionFromAdmin' starting 'in-barrier' execution. 2023-11-28 03:55:55,546 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-28 03:55:55,548 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-28 03:55:55,548 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-28 03:55:55,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-28 03:55:55,548 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionFromAdmin' received 'reached' from coordinator. 2023-11-28 03:55:55,548 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionFromAdmin' locally completed 2023-11-28 03:55:55,548 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionFromAdmin' completed for member 'jenkins-hbase4.apache.org,41203,1701143600653' in zk 2023-11-28 03:55:55,549 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:55,549 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-28 03:55:55,550 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionFromAdmin' has notified controller of completion 2023-11-28 03:55:55,550 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:55,550 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:55:55,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:55,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:55:55,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:55:55,550 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionFromAdmin' completed. 2023-11-28 03:55:55,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:55:55,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:55:55,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-11-28 03:55:55,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:55,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:55:55,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-11-28 03:55:55,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:55,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionFromAdmin' member 'jenkins-hbase4.apache.org,41203,1701143600653': 2023-11-28 03:55:55,554 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' released barrier for procedure'testMajorCompactionFromAdmin', counting down latch. Waiting for 0 more 2023-11-28 03:55:55,554 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionFromAdmin' execution completed 2023-11-28 03:55:55,554 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-28 03:55:55,554 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-28 03:55:55,554 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionFromAdmin 2023-11-28 03:55:55,554 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionFromAdminincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-28 03:55:55,557 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-28 03:55:55,557 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:55:55,557 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-28 03:55:55,557 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-28 03:55:55,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-28 03:55:55,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-28 03:55:55,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:55:55,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:55:55,558 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:55:55,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:55:55,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-28 03:55:55,558 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:55,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:55:55,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-11-28 03:55:55,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:55:55,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-11-28 03:55:55,560 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:55,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:55,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:55:55,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-11-28 03:55:55,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:55,563 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:55,563 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-11-28 03:55:55,563 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-11-28 03:55:55,563 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:55,563 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-28 03:55:55,563 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-28 03:55:55,563 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-28 03:55:55,563 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionFromAdmin' 2023-11-28 03:55:55,563 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:55:55,563 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-28 03:55:55,563 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:55:55,564 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:55:55,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:55:55,564 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:55:55,564 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionFromAdmin'' to complete. (max 300000 ms per retry) 2023-11-28 03:55:55,564 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(399): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin because node does not exist (not an error) 2023-11-28 03:55:55,564 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-28 03:55:55,564 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:55:55,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:55:55,664 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-28 03:55:55,665 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-28 03:55:55,666 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41203] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,,1701143750499.b2248d1fbdc8030e0b590971a233e4f2. with WAL disabled. Data may be lost in the event of a crash. 2023-11-28 03:55:55,666 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41203] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,1,1701143750499.6442e6cb874dc41685a9cdfd56a2179e. with WAL disabled. Data may be lost in the event of a crash. 2023-11-28 03:55:55,666 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41203] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,2,1701143750499.45c07d545bdbc2f92898a848086adde0. with WAL disabled. Data may be lost in the event of a crash. 2023-11-28 03:55:55,668 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-28 03:55:55,670 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionFromAdmin was in running list but was completed. Accepting new attempt. 2023-11-28 03:55:55,671 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionFromAdmin 2023-11-28 03:55:55,671 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionFromAdmin' 2023-11-28 03:55:55,671 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:55:55,671 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionFromAdmin' starting 'acquire' 2023-11-28 03:55:55,671 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionFromAdmin', kicking off acquire phase on members. 2023-11-28 03:55:55,671 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-28 03:55:55,671 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-11-28 03:55:55,673 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:55:55,673 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:55,674 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:55:55,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:55:55,674 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:55,674 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-28 03:55:55,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-11-28 03:55:55,674 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-28 03:55:55,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-28 03:55:55,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-11-28 03:55:55,675 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionFromAdmin 2023-11-28 03:55:55,675 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionFromAdmin is still present, removing 2023-11-28 03:55:55,675 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionFromAdmin 2023-11-28 03:55:55,675 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionFromAdmin' with timeout 60000ms 2023-11-28 03:55:55,675 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:55:55,675 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionFromAdmin' starting 'acquire' stage 2023-11-28 03:55:55,675 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,,1701143750499.b2248d1fbdc8030e0b590971a233e4f2. 2023-11-28 03:55:55,675 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,,1701143750499.b2248d1fbdc8030e0b590971a233e4f2. started... 2023-11-28 03:55:55,675 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-11-28 03:55:55,676 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-28 03:55:55,676 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,2,1701143750499.45c07d545bdbc2f92898a848086adde0. 2023-11-28 03:55:55,676 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,1,1701143750499.6442e6cb874dc41685a9cdfd56a2179e. 2023-11-28 03:55:55,676 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing b2248d1fbdc8030e0b590971a233e4f2 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-11-28 03:55:55,676 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,1,1701143750499.6442e6cb874dc41685a9cdfd56a2179e. started... 2023-11-28 03:55:55,676 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,2,1701143750499.45c07d545bdbc2f92898a848086adde0. started... 2023-11-28 03:55:55,676 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 6442e6cb874dc41685a9cdfd56a2179e 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-11-28 03:55:55,676 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 45c07d545bdbc2f92898a848086adde0 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-11-28 03:55:55,732 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20231128269d1ee93daf4639a2d7afe76c42b6e8 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20231128269d1ee93daf4639a2d7afe76c42b6e8 2023-11-28 03:55:55,736 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023112877384403ba994964ba5ad15627e0685f to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e2023112877384403ba994964ba5ad15627e0685f 2023-11-28 03:55:55,737 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202311287b9e2b69b5114b1cacb60815355bcd8f to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202311287b9e2b69b5114b1cacb60815355bcd8f 2023-11-28 03:55:55,755 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/6442e6cb874dc41685a9cdfd56a2179e/.tmp/family1/12c5607727284891a9470d96da7a3139 2023-11-28 03:55:56,147 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/45c07d545bdbc2f92898a848086adde0/.tmp/family1/87503fab1d434ed0b43de9e24f03107f 2023-11-28 03:55:56,155 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/b2248d1fbdc8030e0b590971a233e4f2/.tmp/family1/92e8aa53de844aa2b345f8783a2dffe9 2023-11-28 03:55:56,189 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20231128f2fb46537d7e4f5c8437b21efb7e64d4 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c4ca4238a0b923820dcc509a6f75849b20231128f2fb46537d7e4f5c8437b21efb7e64d4 2023-11-28 03:55:56,585 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023112872888a4b2a5b46159132a53d66dc2743 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c81e728d9d4c2f636f067f89cc14862c2023112872888a4b2a5b46159132a53d66dc2743 2023-11-28 03:55:56,594 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20231128a6aef3bcb8fc42699a2a96aacdcace96 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/d41d8cd98f00b204e9800998ecf8427e20231128a6aef3bcb8fc42699a2a96aacdcace96 2023-11-28 03:55:56,600 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/6442e6cb874dc41685a9cdfd56a2179e/.tmp/family2/cb40fdef93ab451aa81e8d1e4efd34be 2023-11-28 03:55:56,606 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/b2248d1fbdc8030e0b590971a233e4f2/.tmp/family2/893d2ce61cef482586f268f04bbb4191 2023-11-28 03:55:56,607 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/6442e6cb874dc41685a9cdfd56a2179e/.tmp/family1/12c5607727284891a9470d96da7a3139 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/6442e6cb874dc41685a9cdfd56a2179e/family1/12c5607727284891a9470d96da7a3139 2023-11-28 03:55:56,607 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/45c07d545bdbc2f92898a848086adde0/.tmp/family2/c3cb4c1d64334b70acd5e8ac92b92396 2023-11-28 03:55:56,614 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/45c07d545bdbc2f92898a848086adde0/.tmp/family1/87503fab1d434ed0b43de9e24f03107f as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/45c07d545bdbc2f92898a848086adde0/family1/87503fab1d434ed0b43de9e24f03107f 2023-11-28 03:55:56,618 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/b2248d1fbdc8030e0b590971a233e4f2/.tmp/family1/92e8aa53de844aa2b345f8783a2dffe9 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/b2248d1fbdc8030e0b590971a233e4f2/family1/92e8aa53de844aa2b345f8783a2dffe9 2023-11-28 03:55:56,658 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/b2248d1fbdc8030e0b590971a233e4f2/family1/92e8aa53de844aa2b345f8783a2dffe9, entries=4, sequenceid=13, filesize=5.5 K 2023-11-28 03:55:56,659 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/b2248d1fbdc8030e0b590971a233e4f2/.tmp/family2/893d2ce61cef482586f268f04bbb4191 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/b2248d1fbdc8030e0b590971a233e4f2/family2/893d2ce61cef482586f268f04bbb4191 2023-11-28 03:55:56,661 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/6442e6cb874dc41685a9cdfd56a2179e/family1/12c5607727284891a9470d96da7a3139, entries=4, sequenceid=13, filesize=5.5 K 2023-11-28 03:55:56,667 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/6442e6cb874dc41685a9cdfd56a2179e/.tmp/family2/cb40fdef93ab451aa81e8d1e4efd34be as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/6442e6cb874dc41685a9cdfd56a2179e/family2/cb40fdef93ab451aa81e8d1e4efd34be 2023-11-28 03:55:56,669 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/45c07d545bdbc2f92898a848086adde0/family1/87503fab1d434ed0b43de9e24f03107f, entries=4, sequenceid=13, filesize=5.5 K 2023-11-28 03:55:56,671 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/45c07d545bdbc2f92898a848086adde0/.tmp/family2/c3cb4c1d64334b70acd5e8ac92b92396 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/45c07d545bdbc2f92898a848086adde0/family2/c3cb4c1d64334b70acd5e8ac92b92396 2023-11-28 03:55:56,673 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/b2248d1fbdc8030e0b590971a233e4f2/family2/893d2ce61cef482586f268f04bbb4191, entries=2, sequenceid=13, filesize=5.1 K 2023-11-28 03:55:56,675 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for b2248d1fbdc8030e0b590971a233e4f2 in 999ms, sequenceid=13, compaction requested=true 2023-11-28 03:55:56,675 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for b2248d1fbdc8030e0b590971a233e4f2: 2023-11-28 03:55:56,675 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,,1701143750499.b2248d1fbdc8030e0b590971a233e4f2. 2023-11-28 03:55:56,675 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-11-28 03:55:56,676 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/6442e6cb874dc41685a9cdfd56a2179e/family2/cb40fdef93ab451aa81e8d1e4efd34be, entries=2, sequenceid=13, filesize=5.1 K 2023-11-28 03:55:56,676 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 6442e6cb874dc41685a9cdfd56a2179e in 1000ms, sequenceid=13, compaction requested=true 2023-11-28 03:55:56,677 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 6442e6cb874dc41685a9cdfd56a2179e: 2023-11-28 03:55:56,677 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,1,1701143750499.6442e6cb874dc41685a9cdfd56a2179e. 2023-11-28 03:55:56,677 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-11-28 03:55:56,678 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/45c07d545bdbc2f92898a848086adde0/family2/c3cb4c1d64334b70acd5e8ac92b92396, entries=2, sequenceid=13, filesize=5.1 K 2023-11-28 03:55:56,679 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 45c07d545bdbc2f92898a848086adde0 in 1003ms, sequenceid=13, compaction requested=true 2023-11-28 03:55:56,679 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 45c07d545bdbc2f92898a848086adde0: 2023-11-28 03:55:56,679 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,2,1701143750499.45c07d545bdbc2f92898a848086adde0. 2023-11-28 03:55:56,679 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-11-28 03:55:56,679 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-11-28 03:55:56,679 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:56,679 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionFromAdmin' locally acquired 2023-11-28 03:55:56,679 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure (testMajorCompactionFromAdmin) in zk 2023-11-28 03:55:56,681 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:56,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:56,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:55:56,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:55:56,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:55:56,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:55:56,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-11-28 03:55:56,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:56,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:55:56,684 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure 'testMajorCompactionFromAdmin' on coordinator 2023-11-28 03:55:56,684 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1dfc2b8a[Count = 0] remaining members to acquire global barrier 2023-11-28 03:55:56,684 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionFromAdmin' starting 'in-barrier' execution. 2023-11-28 03:55:56,684 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-28 03:55:56,686 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-28 03:55:56,687 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] zookeeper.ZKUtil(163): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-28 03:55:56,687 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-28 03:55:56,687 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionFromAdmin' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-28 03:55:56,687 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionFromAdmin' received 'reached' from coordinator. 2023-11-28 03:55:56,687 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionFromAdmin' locally completed 2023-11-28 03:55:56,687 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionFromAdmin' completed for member 'jenkins-hbase4.apache.org,41203,1701143600653' in zk 2023-11-28 03:55:56,687 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:56,687 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-28 03:55:56,688 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionFromAdmin' has notified controller of completion 2023-11-28 03:55:56,688 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:55:56,688 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionFromAdmin' completed. 2023-11-28 03:55:56,688 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:56,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:56,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:55:56,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:55:56,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:55:56,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:55:56,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-11-28 03:55:56,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:56,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:55:56,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-11-28 03:55:56,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:56,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionFromAdmin' member 'jenkins-hbase4.apache.org,41203,1701143600653': 2023-11-28 03:55:56,693 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' released barrier for procedure'testMajorCompactionFromAdmin', counting down latch. Waiting for 0 more 2023-11-28 03:55:56,693 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionFromAdmin' execution completed 2023-11-28 03:55:56,693 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-28 03:55:56,693 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-28 03:55:56,693 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionFromAdmin 2023-11-28 03:55:56,693 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionFromAdminincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-28 03:55:56,696 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-28 03:55:56,696 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-28 03:55:56,696 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-28 03:55:56,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-28 03:55:56,696 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:55:56,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-28 03:55:56,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:55:56,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:55:56,697 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:55:56,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:55:56,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:55:56,697 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:56,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-28 03:55:56,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-11-28 03:55:56,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:55:56,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-11-28 03:55:56,701 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:56,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:56,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:55:56,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-11-28 03:55:56,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:56,706 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:56,706 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:55:56,706 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-11-28 03:55:56,706 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:55:56,706 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionFromAdmin' 2023-11-28 03:55:56,707 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-28 03:55:56,707 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:55:56,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:55:56,706 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-28 03:55:56,707 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:55:56,706 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-11-28 03:55:56,707 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:56,707 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-28 03:55:56,707 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-28 03:55:56,707 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-28 03:55:56,707 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionFromAdmin'' to complete. (max 300000 ms per retry) 2023-11-28 03:55:56,707 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-28 03:55:56,707 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:55:56,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:55:56,807 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-28 03:55:56,808 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-28 03:55:56,809 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41203] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,,1701143750499.b2248d1fbdc8030e0b590971a233e4f2. with WAL disabled. Data may be lost in the event of a crash. 2023-11-28 03:55:56,809 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41203] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,1,1701143750499.6442e6cb874dc41685a9cdfd56a2179e. with WAL disabled. Data may be lost in the event of a crash. 2023-11-28 03:55:56,809 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41203] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,2,1701143750499.45c07d545bdbc2f92898a848086adde0. with WAL disabled. Data may be lost in the event of a crash. 2023-11-28 03:55:56,812 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-28 03:55:56,813 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionFromAdmin was in running list but was completed. Accepting new attempt. 2023-11-28 03:55:56,813 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionFromAdmin 2023-11-28 03:55:56,813 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionFromAdmin' 2023-11-28 03:55:56,813 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:55:56,813 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionFromAdmin' starting 'acquire' 2023-11-28 03:55:56,813 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionFromAdmin', kicking off acquire phase on members. 2023-11-28 03:55:56,814 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-28 03:55:56,814 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-11-28 03:55:56,815 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:56,815 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:55:56,815 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:55:56,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:55:56,815 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:56,815 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-28 03:55:56,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-11-28 03:55:56,816 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-28 03:55:56,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-28 03:55:56,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-11-28 03:55:56,816 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionFromAdmin 2023-11-28 03:55:56,816 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionFromAdmin is still present, removing 2023-11-28 03:55:56,816 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionFromAdmin 2023-11-28 03:55:56,816 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionFromAdmin' with timeout 60000ms 2023-11-28 03:55:56,817 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:55:56,817 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionFromAdmin' starting 'acquire' stage 2023-11-28 03:55:56,817 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,,1701143750499.b2248d1fbdc8030e0b590971a233e4f2. 2023-11-28 03:55:56,817 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-11-28 03:55:56,817 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-28 03:55:56,817 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,1,1701143750499.6442e6cb874dc41685a9cdfd56a2179e. 2023-11-28 03:55:56,817 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,,1701143750499.b2248d1fbdc8030e0b590971a233e4f2. started... 2023-11-28 03:55:56,817 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,2,1701143750499.45c07d545bdbc2f92898a848086adde0. 2023-11-28 03:55:56,817 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,1,1701143750499.6442e6cb874dc41685a9cdfd56a2179e. started... 2023-11-28 03:55:56,817 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,2,1701143750499.45c07d545bdbc2f92898a848086adde0. started... 2023-11-28 03:55:56,817 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing b2248d1fbdc8030e0b590971a233e4f2 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-11-28 03:55:56,817 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 6442e6cb874dc41685a9cdfd56a2179e 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-11-28 03:55:56,817 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 45c07d545bdbc2f92898a848086adde0 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-11-28 03:55:56,843 INFO [Finalizer] client.ConnectionImplementation(1973): Closing master protocol: MasterService 2023-11-28 03:55:56,843 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x5abbae49 to 127.0.0.1:55641 2023-11-28 03:55:56,843 DEBUG [Finalizer] ipc.AbstractRpcClient(489): Stopping rpc client 2023-11-28 03:55:56,844 INFO [Finalizer] client.ConnectionImplementation(1973): Closing master protocol: MasterService 2023-11-28 03:55:56,844 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x2d3c06be to 127.0.0.1:55641 2023-11-28 03:55:56,844 DEBUG [Finalizer] ipc.AbstractRpcClient(489): Stopping rpc client 2023-11-28 03:55:56,885 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20231128ab5a2f26563840708cd763f169fb8949 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20231128ab5a2f26563840708cd763f169fb8949 2023-11-28 03:55:57,238 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-11-28 03:55:57,286 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202311283099f69205b346b6af1c1bb20342fac9 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202311283099f69205b346b6af1c1bb20342fac9 2023-11-28 03:55:57,288 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202311287219570b249b48c88ca44d108b53d952 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202311287219570b249b48c88ca44d108b53d952 2023-11-28 03:55:57,298 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/b2248d1fbdc8030e0b590971a233e4f2/.tmp/family1/d3173fe5a4a74c4588417d5807a19fb9 2023-11-28 03:55:57,331 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023112873d3fc2c340c47469e1773c54c15ea5f to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/d41d8cd98f00b204e9800998ecf8427e2023112873d3fc2c340c47469e1773c54c15ea5f 2023-11-28 03:55:57,335 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/b2248d1fbdc8030e0b590971a233e4f2/.tmp/family2/7b232c706cb24c73abff650046cff7cd 2023-11-28 03:55:57,337 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/b2248d1fbdc8030e0b590971a233e4f2/.tmp/family1/d3173fe5a4a74c4588417d5807a19fb9 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/b2248d1fbdc8030e0b590971a233e4f2/family1/d3173fe5a4a74c4588417d5807a19fb9 2023-11-28 03:55:57,362 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/b2248d1fbdc8030e0b590971a233e4f2/family1/d3173fe5a4a74c4588417d5807a19fb9, entries=4, sequenceid=17, filesize=5.5 K 2023-11-28 03:55:57,363 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/b2248d1fbdc8030e0b590971a233e4f2/.tmp/family2/7b232c706cb24c73abff650046cff7cd as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/b2248d1fbdc8030e0b590971a233e4f2/family2/7b232c706cb24c73abff650046cff7cd 2023-11-28 03:55:57,365 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/b2248d1fbdc8030e0b590971a233e4f2/family2/7b232c706cb24c73abff650046cff7cd, entries=2, sequenceid=17, filesize=5.1 K 2023-11-28 03:55:57,366 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for b2248d1fbdc8030e0b590971a233e4f2 in 549ms, sequenceid=17, compaction requested=true 2023-11-28 03:55:57,366 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for b2248d1fbdc8030e0b590971a233e4f2: 2023-11-28 03:55:57,366 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,,1701143750499.b2248d1fbdc8030e0b590971a233e4f2. 2023-11-28 03:55:57,366 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-11-28 03:55:57,701 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/6442e6cb874dc41685a9cdfd56a2179e/.tmp/family1/8a3b1e03086d46d7baaa683499b39657 2023-11-28 03:55:57,703 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/45c07d545bdbc2f92898a848086adde0/.tmp/family1/4c1a72de1fea4afd9e4011336700f7a9 2023-11-28 03:55:57,735 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202311281271303f89204b5a93f6026c3f278870 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c81e728d9d4c2f636f067f89cc14862c202311281271303f89204b5a93f6026c3f278870 2023-11-28 03:55:57,738 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/45c07d545bdbc2f92898a848086adde0/.tmp/family2/85f8ceb20cb54d59a8050b35775fb4cf 2023-11-28 03:55:57,745 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/45c07d545bdbc2f92898a848086adde0/.tmp/family1/4c1a72de1fea4afd9e4011336700f7a9 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/45c07d545bdbc2f92898a848086adde0/family1/4c1a72de1fea4afd9e4011336700f7a9 2023-11-28 03:55:57,779 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/45c07d545bdbc2f92898a848086adde0/family1/4c1a72de1fea4afd9e4011336700f7a9, entries=4, sequenceid=17, filesize=5.5 K 2023-11-28 03:55:57,780 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/45c07d545bdbc2f92898a848086adde0/.tmp/family2/85f8ceb20cb54d59a8050b35775fb4cf as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/45c07d545bdbc2f92898a848086adde0/family2/85f8ceb20cb54d59a8050b35775fb4cf 2023-11-28 03:55:57,784 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/45c07d545bdbc2f92898a848086adde0/family2/85f8ceb20cb54d59a8050b35775fb4cf, entries=2, sequenceid=17, filesize=5.1 K 2023-11-28 03:55:57,785 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 45c07d545bdbc2f92898a848086adde0 in 967ms, sequenceid=17, compaction requested=true 2023-11-28 03:55:57,785 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 45c07d545bdbc2f92898a848086adde0: 2023-11-28 03:55:57,785 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,2,1701143750499.45c07d545bdbc2f92898a848086adde0. 2023-11-28 03:55:57,785 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-11-28 03:55:58,135 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20231128047afff825414a5cb59aa38351b8435b to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c4ca4238a0b923820dcc509a6f75849b20231128047afff825414a5cb59aa38351b8435b 2023-11-28 03:55:58,541 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/6442e6cb874dc41685a9cdfd56a2179e/.tmp/family2/1199315641a546cc8b3fc6ee2ec00b6b 2023-11-28 03:55:58,546 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/6442e6cb874dc41685a9cdfd56a2179e/.tmp/family1/8a3b1e03086d46d7baaa683499b39657 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/6442e6cb874dc41685a9cdfd56a2179e/family1/8a3b1e03086d46d7baaa683499b39657 2023-11-28 03:55:58,581 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/6442e6cb874dc41685a9cdfd56a2179e/family1/8a3b1e03086d46d7baaa683499b39657, entries=4, sequenceid=17, filesize=5.5 K 2023-11-28 03:55:58,582 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/6442e6cb874dc41685a9cdfd56a2179e/.tmp/family2/1199315641a546cc8b3fc6ee2ec00b6b as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/6442e6cb874dc41685a9cdfd56a2179e/family2/1199315641a546cc8b3fc6ee2ec00b6b 2023-11-28 03:55:58,586 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/6442e6cb874dc41685a9cdfd56a2179e/family2/1199315641a546cc8b3fc6ee2ec00b6b, entries=2, sequenceid=17, filesize=5.1 K 2023-11-28 03:55:58,587 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 6442e6cb874dc41685a9cdfd56a2179e in 1770ms, sequenceid=17, compaction requested=true 2023-11-28 03:55:58,587 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 6442e6cb874dc41685a9cdfd56a2179e: 2023-11-28 03:55:58,587 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,1,1701143750499.6442e6cb874dc41685a9cdfd56a2179e. 2023-11-28 03:55:58,587 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-11-28 03:55:58,588 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-11-28 03:55:58,588 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:58,588 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionFromAdmin' locally acquired 2023-11-28 03:55:58,588 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure (testMajorCompactionFromAdmin) in zk 2023-11-28 03:55:58,591 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:58,591 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-28 03:55:58,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:58,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:55:58,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:55:58,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:55:58,592 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-28 03:55:58,592 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionFromAdmin' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-28 03:55:58,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:55:58,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-11-28 03:55:58,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:58,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:55:58,594 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure 'testMajorCompactionFromAdmin' on coordinator 2023-11-28 03:55:58,594 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@76baaea9[Count = 0] remaining members to acquire global barrier 2023-11-28 03:55:58,594 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionFromAdmin' starting 'in-barrier' execution. 2023-11-28 03:55:58,594 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-28 03:55:58,602 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-28 03:55:58,602 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-28 03:55:58,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-28 03:55:58,602 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionFromAdmin' received 'reached' from coordinator. 2023-11-28 03:55:58,602 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionFromAdmin' locally completed 2023-11-28 03:55:58,602 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionFromAdmin' completed for member 'jenkins-hbase4.apache.org,41203,1701143600653' in zk 2023-11-28 03:55:58,602 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:58,602 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-28 03:55:58,604 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionFromAdmin' has notified controller of completion 2023-11-28 03:55:58,604 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:58,604 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:55:58,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:58,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:55:58,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:55:58,604 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionFromAdmin' completed. 2023-11-28 03:55:58,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:55:58,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:55:58,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-11-28 03:55:58,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:58,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:55:58,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-11-28 03:55:58,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:58,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionFromAdmin' member 'jenkins-hbase4.apache.org,41203,1701143600653': 2023-11-28 03:55:58,606 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' released barrier for procedure'testMajorCompactionFromAdmin', counting down latch. Waiting for 0 more 2023-11-28 03:55:58,606 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionFromAdmin' execution completed 2023-11-28 03:55:58,606 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-28 03:55:58,606 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-28 03:55:58,606 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionFromAdmin 2023-11-28 03:55:58,607 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionFromAdminincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-28 03:55:58,610 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-28 03:55:58,610 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-28 03:55:58,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-28 03:55:58,610 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-28 03:55:58,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:55:58,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:55:58,610 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:55:58,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-28 03:55:58,610 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:55:58,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:55:58,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:55:58,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-28 03:55:58,611 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:58,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-11-28 03:55:58,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:55:58,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-11-28 03:55:58,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:58,612 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:58,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:55:58,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-11-28 03:55:58,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:58,617 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:58,617 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-11-28 03:55:58,617 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-11-28 03:55:58,617 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:58,618 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionFromAdmin' 2023-11-28 03:55:58,618 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-28 03:55:58,618 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-28 03:55:58,618 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-28 03:55:58,618 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-28 03:55:58,618 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:55:58,618 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionFromAdmin'' to complete. (max 300000 ms per retry) 2023-11-28 03:55:58,618 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-28 03:55:58,619 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:55:58,619 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:55:58,619 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:55:58,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:55:58,619 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(399): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin because node does not exist (not an error) 2023-11-28 03:55:58,620 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:55:58,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:55:58,718 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-28 03:55:58,719 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-28 03:55:58,731 DEBUG [Listener at localhost/42853] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-11-28 03:55:58,742 DEBUG [Listener at localhost/42853] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-11-28 03:55:58,752 DEBUG [Listener at localhost/42853] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-11-28 03:55:58,753 DEBUG [Listener at localhost/42853] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-11-28 03:55:58,855 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-28 03:55:58,866 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionFromAdmin was in running list but was completed. Accepting new attempt. 2023-11-28 03:55:58,866 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionFromAdmin 2023-11-28 03:55:58,866 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionFromAdmin' 2023-11-28 03:55:58,866 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:55:58,866 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionFromAdmin' starting 'acquire' 2023-11-28 03:55:58,866 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionFromAdmin', kicking off acquire phase on members. 2023-11-28 03:55:58,867 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-28 03:55:58,867 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-11-28 03:55:58,868 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:55:58,868 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:58,868 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:55:58,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:55:58,869 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:55:58,869 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-28 03:55:58,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-11-28 03:55:58,869 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-28 03:55:58,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-28 03:55:58,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-11-28 03:55:58,870 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionFromAdmin 2023-11-28 03:55:58,872 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionFromAdmin is still present, removing 2023-11-28 03:55:58,872 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionFromAdmin 2023-11-28 03:55:58,872 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionFromAdmin' with timeout 60000ms 2023-11-28 03:55:58,872 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-28 03:55:58,872 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionFromAdmin' starting 'acquire' stage 2023-11-28 03:55:58,873 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,,1701143750499.b2248d1fbdc8030e0b590971a233e4f2. 2023-11-28 03:55:58,873 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,,1701143750499.b2248d1fbdc8030e0b590971a233e4f2. started... 2023-11-28 03:55:58,873 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing b2248d1fbdc8030e0b590971a233e4f2 2/2 column families, dataSize=138 B heapSize=944 B 2023-11-28 03:55:58,873 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,2,1701143750499.45c07d545bdbc2f92898a848086adde0. 2023-11-28 03:55:58,873 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,2,1701143750499.45c07d545bdbc2f92898a848086adde0. started... 2023-11-28 03:55:58,873 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 45c07d545bdbc2f92898a848086adde0 2/2 column families, dataSize=138 B heapSize=944 B 2023-11-28 03:55:58,876 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-11-28 03:55:58,876 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-28 03:55:58,877 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,1,1701143750499.6442e6cb874dc41685a9cdfd56a2179e. 2023-11-28 03:55:58,878 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,1,1701143750499.6442e6cb874dc41685a9cdfd56a2179e. started... 2023-11-28 03:55:58,878 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 6442e6cb874dc41685a9cdfd56a2179e 2/2 column families, dataSize=138 B heapSize=944 B 2023-11-28 03:55:58,930 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/45c07d545bdbc2f92898a848086adde0/.tmp/family1/06faf74b932c4bbb94352d7c0a82df0d 2023-11-28 03:55:58,941 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/6442e6cb874dc41685a9cdfd56a2179e/.tmp/family1/4de8d1e3f74647ccb14a9dc5a8fcc3e5 2023-11-28 03:55:58,966 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 06faf74b932c4bbb94352d7c0a82df0d 2023-11-28 03:55:58,978 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 4de8d1e3f74647ccb14a9dc5a8fcc3e5 2023-11-28 03:55:59,416 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/45c07d545bdbc2f92898a848086adde0/.tmp/family2/cbdc7b405feb405cb4818757c7a03793 2023-11-28 03:55:59,419 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for cbdc7b405feb405cb4818757c7a03793 2023-11-28 03:55:59,419 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/45c07d545bdbc2f92898a848086adde0/.tmp/family1/06faf74b932c4bbb94352d7c0a82df0d as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/45c07d545bdbc2f92898a848086adde0/family1/06faf74b932c4bbb94352d7c0a82df0d 2023-11-28 03:55:59,432 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/6442e6cb874dc41685a9cdfd56a2179e/.tmp/family2/bf321675bb1743878e838aadec3428ba 2023-11-28 03:55:59,434 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for bf321675bb1743878e838aadec3428ba 2023-11-28 03:55:59,435 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/6442e6cb874dc41685a9cdfd56a2179e/.tmp/family1/4de8d1e3f74647ccb14a9dc5a8fcc3e5 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/6442e6cb874dc41685a9cdfd56a2179e/family1/4de8d1e3f74647ccb14a9dc5a8fcc3e5 2023-11-28 03:55:59,445 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 06faf74b932c4bbb94352d7c0a82df0d 2023-11-28 03:55:59,445 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/45c07d545bdbc2f92898a848086adde0/family1/06faf74b932c4bbb94352d7c0a82df0d, entries=3, sequenceid=23, filesize=5.2 K 2023-11-28 03:55:59,446 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/45c07d545bdbc2f92898a848086adde0/.tmp/family2/cbdc7b405feb405cb4818757c7a03793 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/45c07d545bdbc2f92898a848086adde0/family2/cbdc7b405feb405cb4818757c7a03793 2023-11-28 03:55:59,449 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for cbdc7b405feb405cb4818757c7a03793 2023-11-28 03:55:59,450 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/45c07d545bdbc2f92898a848086adde0/family2/cbdc7b405feb405cb4818757c7a03793, entries=1, sequenceid=23, filesize=5.0 K 2023-11-28 03:55:59,450 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 45c07d545bdbc2f92898a848086adde0 in 577ms, sequenceid=23, compaction requested=true 2023-11-28 03:55:59,450 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 45c07d545bdbc2f92898a848086adde0: 2023-11-28 03:55:59,450 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,2,1701143750499.45c07d545bdbc2f92898a848086adde0. 2023-11-28 03:55:59,451 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-11-28 03:55:59,457 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 4de8d1e3f74647ccb14a9dc5a8fcc3e5 2023-11-28 03:55:59,457 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/6442e6cb874dc41685a9cdfd56a2179e/family1/4de8d1e3f74647ccb14a9dc5a8fcc3e5, entries=3, sequenceid=23, filesize=5.2 K 2023-11-28 03:55:59,458 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/6442e6cb874dc41685a9cdfd56a2179e/.tmp/family2/bf321675bb1743878e838aadec3428ba as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/6442e6cb874dc41685a9cdfd56a2179e/family2/bf321675bb1743878e838aadec3428ba 2023-11-28 03:55:59,461 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for bf321675bb1743878e838aadec3428ba 2023-11-28 03:55:59,461 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/6442e6cb874dc41685a9cdfd56a2179e/family2/bf321675bb1743878e838aadec3428ba, entries=1, sequenceid=23, filesize=5.0 K 2023-11-28 03:55:59,462 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 6442e6cb874dc41685a9cdfd56a2179e in 584ms, sequenceid=23, compaction requested=true 2023-11-28 03:55:59,462 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 6442e6cb874dc41685a9cdfd56a2179e: 2023-11-28 03:55:59,462 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,1,1701143750499.6442e6cb874dc41685a9cdfd56a2179e. 2023-11-28 03:55:59,462 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-11-28 03:55:59,739 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/b2248d1fbdc8030e0b590971a233e4f2/.tmp/family1/c3bd9b7c3f444c429cf43747e55a2c02 2023-11-28 03:55:59,762 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c3bd9b7c3f444c429cf43747e55a2c02 2023-11-28 03:56:00,573 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/b2248d1fbdc8030e0b590971a233e4f2/.tmp/family2/79f1ac856c3d45e8ab4adcf47abb1911 2023-11-28 03:56:00,576 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 79f1ac856c3d45e8ab4adcf47abb1911 2023-11-28 03:56:00,577 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/b2248d1fbdc8030e0b590971a233e4f2/.tmp/family1/c3bd9b7c3f444c429cf43747e55a2c02 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/b2248d1fbdc8030e0b590971a233e4f2/family1/c3bd9b7c3f444c429cf43747e55a2c02 2023-11-28 03:56:00,601 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c3bd9b7c3f444c429cf43747e55a2c02 2023-11-28 03:56:00,601 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/b2248d1fbdc8030e0b590971a233e4f2/family1/c3bd9b7c3f444c429cf43747e55a2c02, entries=3, sequenceid=23, filesize=5.2 K 2023-11-28 03:56:00,601 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/b2248d1fbdc8030e0b590971a233e4f2/.tmp/family2/79f1ac856c3d45e8ab4adcf47abb1911 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/b2248d1fbdc8030e0b590971a233e4f2/family2/79f1ac856c3d45e8ab4adcf47abb1911 2023-11-28 03:56:00,605 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 79f1ac856c3d45e8ab4adcf47abb1911 2023-11-28 03:56:00,605 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/b2248d1fbdc8030e0b590971a233e4f2/family2/79f1ac856c3d45e8ab4adcf47abb1911, entries=1, sequenceid=23, filesize=5.0 K 2023-11-28 03:56:00,605 INFO [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for b2248d1fbdc8030e0b590971a233e4f2 in 1732ms, sequenceid=23, compaction requested=true 2023-11-28 03:56:00,606 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for b2248d1fbdc8030e0b590971a233e4f2: 2023-11-28 03:56:00,606 DEBUG [rs(jenkins-hbase4.apache.org,41203,1701143600653)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,,1701143750499.b2248d1fbdc8030e0b590971a233e4f2. 2023-11-28 03:56:00,606 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-11-28 03:56:00,606 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-11-28 03:56:00,606 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:56:00,606 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionFromAdmin' locally acquired 2023-11-28 03:56:00,606 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure (testMajorCompactionFromAdmin) in zk 2023-11-28 03:56:00,607 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:56:00,607 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-28 03:56:00,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:56:00,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:56:00,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:56:00,608 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-28 03:56:00,608 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionFromAdmin' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-28 03:56:00,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:56:00,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:56:00,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-11-28 03:56:00,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:56:00,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:56:00,609 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41203,1701143600653' joining acquired barrier for procedure 'testMajorCompactionFromAdmin' on coordinator 2023-11-28 03:56:00,609 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@66111532[Count = 0] remaining members to acquire global barrier 2023-11-28 03:56:00,609 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionFromAdmin' starting 'in-barrier' execution. 2023-11-28 03:56:00,609 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-28 03:56:00,611 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-28 03:56:00,611 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-28 03:56:00,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-28 03:56:00,611 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionFromAdmin' received 'reached' from coordinator. 2023-11-28 03:56:00,612 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionFromAdmin' locally completed 2023-11-28 03:56:00,611 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:56:00,612 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-28 03:56:00,612 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionFromAdmin' completed for member 'jenkins-hbase4.apache.org,41203,1701143600653' in zk 2023-11-28 03:56:00,613 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:56:00,613 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionFromAdmin' has notified controller of completion 2023-11-28 03:56:00,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:56:00,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:56:00,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:56:00,613 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:56:00,613 DEBUG [member: 'jenkins-hbase4.apache.org,41203,1701143600653' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionFromAdmin' completed. 2023-11-28 03:56:00,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:56:00,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:56:00,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-11-28 03:56:00,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:56:00,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:56:00,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-11-28 03:56:00,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:56:00,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionFromAdmin' member 'jenkins-hbase4.apache.org,41203,1701143600653': 2023-11-28 03:56:00,615 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41203,1701143600653' released barrier for procedure'testMajorCompactionFromAdmin', counting down latch. Waiting for 0 more 2023-11-28 03:56:00,615 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionFromAdmin' execution completed 2023-11-28 03:56:00,615 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-28 03:56:00,615 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-28 03:56:00,615 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionFromAdmin 2023-11-28 03:56:00,615 INFO [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionFromAdminincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-28 03:56:00,617 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-28 03:56:00,617 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-28 03:56:00,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-28 03:56:00,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-28 03:56:00,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-28 03:56:00,617 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-28 03:56:00,617 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:56:00,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-28 03:56:00,617 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:56:00,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-28 03:56:00,617 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:56:00,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:56:00,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-11-28 03:56:00,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-28 03:56:00,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-28 03:56:00,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-11-28 03:56:00,618 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:56:00,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:56:00,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-28 03:56:00,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-11-28 03:56:00,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:56:00,620 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:56:00,620 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-11-28 03:56:00,620 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-28 03:56:00,620 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-11-28 03:56:00,620 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-28 03:56:00,620 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:56:00,621 DEBUG [(jenkins-hbase4.apache.org,38593,1701143599478)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-28 03:56:00,621 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(399): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin because node does not exist (not an error) 2023-11-28 03:56:00,620 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-28 03:56:00,621 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionFromAdmin' 2023-11-28 03:56:00,621 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-28 03:56:00,621 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-28 03:56:00,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-28 03:56:00,621 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-28 03:56:00,621 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-28 03:56:00,621 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionFromAdmin'' to complete. (max 300000 ms per retry) 2023-11-28 03:56:00,621 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-28 03:56:00,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-28 03:56:00,621 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-28 03:56:00,721 DEBUG [Listener at localhost/42853] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-28 03:56:00,722 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38593] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-28 03:56:00,722 DEBUG [Listener at localhost/42853] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-11-28 03:56:00,722 DEBUG [Listener at localhost/42853] regionserver.HStore(1912): b2248d1fbdc8030e0b590971a233e4f2/family1 is initiating major compaction (all files) 2023-11-28 03:56:00,723 INFO [Listener at localhost/42853] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-11-28 03:56:00,723 INFO [Listener at localhost/42853] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-11-28 03:56:00,723 INFO [Listener at localhost/42853] regionserver.HRegion(2259): Starting compaction of b2248d1fbdc8030e0b590971a233e4f2/family1 in testMajorCompactionFromAdmin,,1701143750499.b2248d1fbdc8030e0b590971a233e4f2. 2023-11-28 03:56:00,724 INFO [Listener at localhost/42853] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/b2248d1fbdc8030e0b590971a233e4f2/family1/c72b173385884f6e8fefb324934c1840, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/b2248d1fbdc8030e0b590971a233e4f2/family1/d4a3e5e991fa43e39a41e6dfc30b3de9, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/b2248d1fbdc8030e0b590971a233e4f2/family1/92e8aa53de844aa2b345f8783a2dffe9, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/b2248d1fbdc8030e0b590971a233e4f2/family1/d3173fe5a4a74c4588417d5807a19fb9, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/b2248d1fbdc8030e0b590971a233e4f2/family1/c3bd9b7c3f444c429cf43747e55a2c02] into tmpdir=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/b2248d1fbdc8030e0b590971a233e4f2/.tmp, totalSize=27.1 K 2023-11-28 03:56:00,724 DEBUG [Listener at localhost/42853] compactions.Compactor(207): Compacting c72b173385884f6e8fefb324934c1840, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1701143752647 2023-11-28 03:56:00,724 DEBUG [Listener at localhost/42853] compactions.Compactor(207): Compacting d4a3e5e991fa43e39a41e6dfc30b3de9, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1701143754565 2023-11-28 03:56:00,725 DEBUG [Listener at localhost/42853] compactions.Compactor(207): Compacting 92e8aa53de844aa2b345f8783a2dffe9, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1701143755666 2023-11-28 03:56:00,725 DEBUG [Listener at localhost/42853] compactions.Compactor(207): Compacting d3173fe5a4a74c4588417d5807a19fb9, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1701143756809 2023-11-28 03:56:00,725 DEBUG [Listener at localhost/42853] compactions.Compactor(207): Compacting c3bd9b7c3f444c429cf43747e55a2c02, keycount=3, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-11-28 03:56:00,733 INFO [Listener at localhost/42853] throttle.PressureAwareThroughputController(145): b2248d1fbdc8030e0b590971a233e4f2#family1#compaction#197 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-11-28 03:56:01,170 INFO [Listener at localhost/42853] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e20231128fc765e646fa04a62a733cbbd7cfd539c_del 2023-11-28 03:56:01,170 INFO [Listener at localhost/42853] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20231128fc765e646fa04a62a733cbbd7cfd539c_del to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20231128fc765e646fa04a62a733cbbd7cfd539c_del 2023-11-28 03:56:02,022 INFO [Listener at localhost/42853] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d75bacbf288c47cb8fc05d46daf8dad9 2023-11-28 03:56:02,023 DEBUG [Listener at localhost/42853] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/b2248d1fbdc8030e0b590971a233e4f2/.tmp/family1/d75bacbf288c47cb8fc05d46daf8dad9 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/b2248d1fbdc8030e0b590971a233e4f2/family1/d75bacbf288c47cb8fc05d46daf8dad9 2023-11-28 03:56:02,044 INFO [Listener at localhost/42853] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d75bacbf288c47cb8fc05d46daf8dad9 2023-11-28 03:56:02,045 INFO [Listener at localhost/42853] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in b2248d1fbdc8030e0b590971a233e4f2/family1 of b2248d1fbdc8030e0b590971a233e4f2 into d75bacbf288c47cb8fc05d46daf8dad9(size=7.0 K), total size for store is 7.0 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-11-28 03:56:02,045 DEBUG [Listener at localhost/42853] regionserver.HRegion(2289): Compaction status journal for b2248d1fbdc8030e0b590971a233e4f2: 2023-11-28 03:56:02,045 DEBUG [Listener at localhost/42853] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-11-28 03:56:02,046 DEBUG [Listener at localhost/42853] regionserver.HStore(1912): b2248d1fbdc8030e0b590971a233e4f2/family2 is initiating major compaction (all files) 2023-11-28 03:56:02,046 INFO [Listener at localhost/42853] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-11-28 03:56:02,046 INFO [Listener at localhost/42853] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-11-28 03:56:02,046 INFO [Listener at localhost/42853] regionserver.HRegion(2259): Starting compaction of b2248d1fbdc8030e0b590971a233e4f2/family2 in testMajorCompactionFromAdmin,,1701143750499.b2248d1fbdc8030e0b590971a233e4f2. 2023-11-28 03:56:02,046 INFO [Listener at localhost/42853] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/b2248d1fbdc8030e0b590971a233e4f2/family2/8df51ba4358b460b86db02295d3ef6d8, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/b2248d1fbdc8030e0b590971a233e4f2/family2/29407608efa54d139b093bd6a751812a, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/b2248d1fbdc8030e0b590971a233e4f2/family2/893d2ce61cef482586f268f04bbb4191, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/b2248d1fbdc8030e0b590971a233e4f2/family2/7b232c706cb24c73abff650046cff7cd, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/b2248d1fbdc8030e0b590971a233e4f2/family2/79f1ac856c3d45e8ab4adcf47abb1911] into tmpdir=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/b2248d1fbdc8030e0b590971a233e4f2/.tmp, totalSize=25.4 K 2023-11-28 03:56:02,047 DEBUG [Listener at localhost/42853] compactions.Compactor(207): Compacting 8df51ba4358b460b86db02295d3ef6d8, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1701143752647 2023-11-28 03:56:02,047 DEBUG [Listener at localhost/42853] compactions.Compactor(207): Compacting 29407608efa54d139b093bd6a751812a, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1701143754565 2023-11-28 03:56:02,047 DEBUG [Listener at localhost/42853] compactions.Compactor(207): Compacting 893d2ce61cef482586f268f04bbb4191, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1701143755666 2023-11-28 03:56:02,048 DEBUG [Listener at localhost/42853] compactions.Compactor(207): Compacting 7b232c706cb24c73abff650046cff7cd, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1701143756809 2023-11-28 03:56:02,048 DEBUG [Listener at localhost/42853] compactions.Compactor(207): Compacting 79f1ac856c3d45e8ab4adcf47abb1911, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-11-28 03:56:02,055 INFO [Listener at localhost/42853] throttle.PressureAwareThroughputController(145): b2248d1fbdc8030e0b590971a233e4f2#family2#compaction#198 average throughput is 0.53 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-11-28 03:56:02,106 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/b2248d1fbdc8030e0b590971a233e4f2/family1/c72b173385884f6e8fefb324934c1840, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/b2248d1fbdc8030e0b590971a233e4f2/family1/d4a3e5e991fa43e39a41e6dfc30b3de9, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/b2248d1fbdc8030e0b590971a233e4f2/family1/92e8aa53de844aa2b345f8783a2dffe9, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/b2248d1fbdc8030e0b590971a233e4f2/family1/d3173fe5a4a74c4588417d5807a19fb9, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/b2248d1fbdc8030e0b590971a233e4f2/family1/c3bd9b7c3f444c429cf43747e55a2c02] to archive 2023-11-28 03:56:02,106 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-28 03:56:02,108 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/b2248d1fbdc8030e0b590971a233e4f2/family1/c72b173385884f6e8fefb324934c1840 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionFromAdmin/b2248d1fbdc8030e0b590971a233e4f2/family1/c72b173385884f6e8fefb324934c1840 2023-11-28 03:56:02,109 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/b2248d1fbdc8030e0b590971a233e4f2/family1/d4a3e5e991fa43e39a41e6dfc30b3de9 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionFromAdmin/b2248d1fbdc8030e0b590971a233e4f2/family1/d4a3e5e991fa43e39a41e6dfc30b3de9 2023-11-28 03:56:02,110 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/b2248d1fbdc8030e0b590971a233e4f2/family1/92e8aa53de844aa2b345f8783a2dffe9 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionFromAdmin/b2248d1fbdc8030e0b590971a233e4f2/family1/92e8aa53de844aa2b345f8783a2dffe9 2023-11-28 03:56:02,111 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/b2248d1fbdc8030e0b590971a233e4f2/family1/d3173fe5a4a74c4588417d5807a19fb9 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionFromAdmin/b2248d1fbdc8030e0b590971a233e4f2/family1/d3173fe5a4a74c4588417d5807a19fb9 2023-11-28 03:56:02,112 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/b2248d1fbdc8030e0b590971a233e4f2/family1/c3bd9b7c3f444c429cf43747e55a2c02 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionFromAdmin/b2248d1fbdc8030e0b590971a233e4f2/family1/c3bd9b7c3f444c429cf43747e55a2c02 2023-11-28 03:56:02,462 INFO [Listener at localhost/42853] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e202311289935d809f78e441198162103f3c121ab_del 2023-11-28 03:56:02,462 INFO [Listener at localhost/42853] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202311289935d809f78e441198162103f3c121ab_del to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/d41d8cd98f00b204e9800998ecf8427e202311289935d809f78e441198162103f3c121ab_del 2023-11-28 03:56:03,276 INFO [Listener at localhost/42853] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e3b2689f13da483981078f53fd767751 2023-11-28 03:56:03,277 DEBUG [Listener at localhost/42853] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/b2248d1fbdc8030e0b590971a233e4f2/.tmp/family2/e3b2689f13da483981078f53fd767751 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/b2248d1fbdc8030e0b590971a233e4f2/family2/e3b2689f13da483981078f53fd767751 2023-11-28 03:56:03,281 INFO [Listener at localhost/42853] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e3b2689f13da483981078f53fd767751 2023-11-28 03:56:03,282 INFO [Listener at localhost/42853] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in b2248d1fbdc8030e0b590971a233e4f2/family2 of b2248d1fbdc8030e0b590971a233e4f2 into e3b2689f13da483981078f53fd767751(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-11-28 03:56:03,282 DEBUG [Listener at localhost/42853] regionserver.HRegion(2289): Compaction status journal for b2248d1fbdc8030e0b590971a233e4f2: 2023-11-28 03:56:03,282 DEBUG [Listener at localhost/42853] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-11-28 03:56:03,282 DEBUG [Listener at localhost/42853] regionserver.HStore(1912): 45c07d545bdbc2f92898a848086adde0/family1 is initiating major compaction (all files) 2023-11-28 03:56:03,282 INFO [Listener at localhost/42853] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-11-28 03:56:03,283 INFO [Listener at localhost/42853] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-11-28 03:56:03,284 INFO [Listener at localhost/42853] regionserver.HRegion(2259): Starting compaction of 45c07d545bdbc2f92898a848086adde0/family1 in testMajorCompactionFromAdmin,2,1701143750499.45c07d545bdbc2f92898a848086adde0. 2023-11-28 03:56:03,284 INFO [Listener at localhost/42853] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/45c07d545bdbc2f92898a848086adde0/family1/32df831f9edc48e0a26c4c619f13ce3b, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/45c07d545bdbc2f92898a848086adde0/family1/421a2ed221404530a8265481bd5e5ad8, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/45c07d545bdbc2f92898a848086adde0/family1/87503fab1d434ed0b43de9e24f03107f, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/45c07d545bdbc2f92898a848086adde0/family1/4c1a72de1fea4afd9e4011336700f7a9, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/45c07d545bdbc2f92898a848086adde0/family1/06faf74b932c4bbb94352d7c0a82df0d] into tmpdir=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/45c07d545bdbc2f92898a848086adde0/.tmp, totalSize=27.1 K 2023-11-28 03:56:03,285 DEBUG [Listener at localhost/42853] compactions.Compactor(207): Compacting 32df831f9edc48e0a26c4c619f13ce3b, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1701143752647 2023-11-28 03:56:03,285 DEBUG [Listener at localhost/42853] compactions.Compactor(207): Compacting 421a2ed221404530a8265481bd5e5ad8, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1701143754565 2023-11-28 03:56:03,285 DEBUG [Listener at localhost/42853] compactions.Compactor(207): Compacting 87503fab1d434ed0b43de9e24f03107f, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1701143755666 2023-11-28 03:56:03,286 DEBUG [Listener at localhost/42853] compactions.Compactor(207): Compacting 4c1a72de1fea4afd9e4011336700f7a9, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1701143756809 2023-11-28 03:56:03,286 DEBUG [Listener at localhost/42853] compactions.Compactor(207): Compacting 06faf74b932c4bbb94352d7c0a82df0d, keycount=3, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-11-28 03:56:03,302 INFO [Listener at localhost/42853] throttle.PressureAwareThroughputController(145): 45c07d545bdbc2f92898a848086adde0#family1#compaction#199 average throughput is 0.19 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-11-28 03:56:03,306 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/b2248d1fbdc8030e0b590971a233e4f2/family2/8df51ba4358b460b86db02295d3ef6d8, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/b2248d1fbdc8030e0b590971a233e4f2/family2/29407608efa54d139b093bd6a751812a, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/b2248d1fbdc8030e0b590971a233e4f2/family2/893d2ce61cef482586f268f04bbb4191, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/b2248d1fbdc8030e0b590971a233e4f2/family2/7b232c706cb24c73abff650046cff7cd, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/b2248d1fbdc8030e0b590971a233e4f2/family2/79f1ac856c3d45e8ab4adcf47abb1911] to archive 2023-11-28 03:56:03,307 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-28 03:56:03,309 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/b2248d1fbdc8030e0b590971a233e4f2/family2/8df51ba4358b460b86db02295d3ef6d8 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionFromAdmin/b2248d1fbdc8030e0b590971a233e4f2/family2/8df51ba4358b460b86db02295d3ef6d8 2023-11-28 03:56:03,311 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/b2248d1fbdc8030e0b590971a233e4f2/family2/29407608efa54d139b093bd6a751812a to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionFromAdmin/b2248d1fbdc8030e0b590971a233e4f2/family2/29407608efa54d139b093bd6a751812a 2023-11-28 03:56:03,312 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/b2248d1fbdc8030e0b590971a233e4f2/family2/893d2ce61cef482586f268f04bbb4191 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionFromAdmin/b2248d1fbdc8030e0b590971a233e4f2/family2/893d2ce61cef482586f268f04bbb4191 2023-11-28 03:56:03,314 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/b2248d1fbdc8030e0b590971a233e4f2/family2/7b232c706cb24c73abff650046cff7cd to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionFromAdmin/b2248d1fbdc8030e0b590971a233e4f2/family2/7b232c706cb24c73abff650046cff7cd 2023-11-28 03:56:03,315 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/b2248d1fbdc8030e0b590971a233e4f2/family2/79f1ac856c3d45e8ab4adcf47abb1911 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionFromAdmin/b2248d1fbdc8030e0b590971a233e4f2/family2/79f1ac856c3d45e8ab4adcf47abb1911 2023-11-28 03:56:03,364 INFO [Listener at localhost/42853] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c20231128cc99133d752c40f5bb896aa9fe2da69e_del 2023-11-28 03:56:03,364 INFO [Listener at localhost/42853] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20231128cc99133d752c40f5bb896aa9fe2da69e_del to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20231128cc99133d752c40f5bb896aa9fe2da69e_del 2023-11-28 03:56:04,236 INFO [Listener at localhost/42853] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for be7d9d034d0b471a9f7aa3b8caa2d7ea 2023-11-28 03:56:04,237 DEBUG [Listener at localhost/42853] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/45c07d545bdbc2f92898a848086adde0/.tmp/family1/be7d9d034d0b471a9f7aa3b8caa2d7ea as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/45c07d545bdbc2f92898a848086adde0/family1/be7d9d034d0b471a9f7aa3b8caa2d7ea 2023-11-28 03:56:04,261 INFO [Listener at localhost/42853] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for be7d9d034d0b471a9f7aa3b8caa2d7ea 2023-11-28 03:56:04,262 INFO [Listener at localhost/42853] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 45c07d545bdbc2f92898a848086adde0/family1 of 45c07d545bdbc2f92898a848086adde0 into be7d9d034d0b471a9f7aa3b8caa2d7ea(size=7.0 K), total size for store is 7.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-11-28 03:56:04,262 DEBUG [Listener at localhost/42853] regionserver.HRegion(2289): Compaction status journal for 45c07d545bdbc2f92898a848086adde0: 2023-11-28 03:56:04,262 DEBUG [Listener at localhost/42853] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-11-28 03:56:04,262 DEBUG [Listener at localhost/42853] regionserver.HStore(1912): 45c07d545bdbc2f92898a848086adde0/family2 is initiating major compaction (all files) 2023-11-28 03:56:04,262 INFO [Listener at localhost/42853] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-11-28 03:56:04,262 INFO [Listener at localhost/42853] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-11-28 03:56:04,263 INFO [Listener at localhost/42853] regionserver.HRegion(2259): Starting compaction of 45c07d545bdbc2f92898a848086adde0/family2 in testMajorCompactionFromAdmin,2,1701143750499.45c07d545bdbc2f92898a848086adde0. 2023-11-28 03:56:04,263 INFO [Listener at localhost/42853] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/45c07d545bdbc2f92898a848086adde0/family2/52498b831e88442095d4a1cccc76b337, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/45c07d545bdbc2f92898a848086adde0/family2/c8f31136ec6c48f58e1c448f21f608fc, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/45c07d545bdbc2f92898a848086adde0/family2/c3cb4c1d64334b70acd5e8ac92b92396, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/45c07d545bdbc2f92898a848086adde0/family2/85f8ceb20cb54d59a8050b35775fb4cf, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/45c07d545bdbc2f92898a848086adde0/family2/cbdc7b405feb405cb4818757c7a03793] into tmpdir=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/45c07d545bdbc2f92898a848086adde0/.tmp, totalSize=25.4 K 2023-11-28 03:56:04,263 DEBUG [Listener at localhost/42853] compactions.Compactor(207): Compacting 52498b831e88442095d4a1cccc76b337, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1701143752647 2023-11-28 03:56:04,264 DEBUG [Listener at localhost/42853] compactions.Compactor(207): Compacting c8f31136ec6c48f58e1c448f21f608fc, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1701143754565 2023-11-28 03:56:04,264 DEBUG [Listener at localhost/42853] compactions.Compactor(207): Compacting c3cb4c1d64334b70acd5e8ac92b92396, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1701143755666 2023-11-28 03:56:04,264 DEBUG [Listener at localhost/42853] compactions.Compactor(207): Compacting 85f8ceb20cb54d59a8050b35775fb4cf, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1701143756809 2023-11-28 03:56:04,264 DEBUG [Listener at localhost/42853] compactions.Compactor(207): Compacting cbdc7b405feb405cb4818757c7a03793, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-11-28 03:56:04,272 INFO [Listener at localhost/42853] throttle.PressureAwareThroughputController(145): 45c07d545bdbc2f92898a848086adde0#family2#compaction#200 average throughput is 0.53 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-11-28 03:56:04,278 INFO [Listener at localhost/42853] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c202311287221678f1e114a00999f13e9c4d95be3_del 2023-11-28 03:56:04,278 INFO [Listener at localhost/42853] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202311287221678f1e114a00999f13e9c4d95be3_del to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c81e728d9d4c2f636f067f89cc14862c202311287221678f1e114a00999f13e9c4d95be3_del 2023-11-28 03:56:04,305 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/45c07d545bdbc2f92898a848086adde0/family1/32df831f9edc48e0a26c4c619f13ce3b, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/45c07d545bdbc2f92898a848086adde0/family1/421a2ed221404530a8265481bd5e5ad8, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/45c07d545bdbc2f92898a848086adde0/family1/87503fab1d434ed0b43de9e24f03107f, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/45c07d545bdbc2f92898a848086adde0/family1/4c1a72de1fea4afd9e4011336700f7a9, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/45c07d545bdbc2f92898a848086adde0/family1/06faf74b932c4bbb94352d7c0a82df0d] to archive 2023-11-28 03:56:04,305 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-28 03:56:04,307 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/45c07d545bdbc2f92898a848086adde0/family1/32df831f9edc48e0a26c4c619f13ce3b to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionFromAdmin/45c07d545bdbc2f92898a848086adde0/family1/32df831f9edc48e0a26c4c619f13ce3b 2023-11-28 03:56:04,308 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/45c07d545bdbc2f92898a848086adde0/family1/421a2ed221404530a8265481bd5e5ad8 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionFromAdmin/45c07d545bdbc2f92898a848086adde0/family1/421a2ed221404530a8265481bd5e5ad8 2023-11-28 03:56:04,308 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/45c07d545bdbc2f92898a848086adde0/family1/87503fab1d434ed0b43de9e24f03107f to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionFromAdmin/45c07d545bdbc2f92898a848086adde0/family1/87503fab1d434ed0b43de9e24f03107f 2023-11-28 03:56:04,309 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/45c07d545bdbc2f92898a848086adde0/family1/4c1a72de1fea4afd9e4011336700f7a9 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionFromAdmin/45c07d545bdbc2f92898a848086adde0/family1/4c1a72de1fea4afd9e4011336700f7a9 2023-11-28 03:56:04,310 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/45c07d545bdbc2f92898a848086adde0/family1/06faf74b932c4bbb94352d7c0a82df0d to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionFromAdmin/45c07d545bdbc2f92898a848086adde0/family1/06faf74b932c4bbb94352d7c0a82df0d 2023-11-28 03:56:05,090 INFO [Listener at localhost/42853] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e862f02bdf0b428dae1ecf55ec5c3751 2023-11-28 03:56:05,091 DEBUG [Listener at localhost/42853] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/45c07d545bdbc2f92898a848086adde0/.tmp/family2/e862f02bdf0b428dae1ecf55ec5c3751 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/45c07d545bdbc2f92898a848086adde0/family2/e862f02bdf0b428dae1ecf55ec5c3751 2023-11-28 03:56:05,095 INFO [Listener at localhost/42853] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e862f02bdf0b428dae1ecf55ec5c3751 2023-11-28 03:56:05,096 INFO [Listener at localhost/42853] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 45c07d545bdbc2f92898a848086adde0/family2 of 45c07d545bdbc2f92898a848086adde0 into e862f02bdf0b428dae1ecf55ec5c3751(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-11-28 03:56:05,096 DEBUG [Listener at localhost/42853] regionserver.HRegion(2289): Compaction status journal for 45c07d545bdbc2f92898a848086adde0: 2023-11-28 03:56:05,096 DEBUG [Listener at localhost/42853] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-11-28 03:56:05,096 DEBUG [Listener at localhost/42853] regionserver.HStore(1912): 6442e6cb874dc41685a9cdfd56a2179e/family1 is initiating major compaction (all files) 2023-11-28 03:56:05,096 INFO [Listener at localhost/42853] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-11-28 03:56:05,096 INFO [Listener at localhost/42853] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-11-28 03:56:05,096 INFO [Listener at localhost/42853] regionserver.HRegion(2259): Starting compaction of 6442e6cb874dc41685a9cdfd56a2179e/family1 in testMajorCompactionFromAdmin,1,1701143750499.6442e6cb874dc41685a9cdfd56a2179e. 2023-11-28 03:56:05,096 INFO [Listener at localhost/42853] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/6442e6cb874dc41685a9cdfd56a2179e/family1/3c668f19e1ce4476b89ff27928db8028, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/6442e6cb874dc41685a9cdfd56a2179e/family1/6dd6971d94d64d53a807de63c42e191f, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/6442e6cb874dc41685a9cdfd56a2179e/family1/12c5607727284891a9470d96da7a3139, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/6442e6cb874dc41685a9cdfd56a2179e/family1/8a3b1e03086d46d7baaa683499b39657, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/6442e6cb874dc41685a9cdfd56a2179e/family1/4de8d1e3f74647ccb14a9dc5a8fcc3e5] into tmpdir=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/6442e6cb874dc41685a9cdfd56a2179e/.tmp, totalSize=27.1 K 2023-11-28 03:56:05,097 DEBUG [Listener at localhost/42853] compactions.Compactor(207): Compacting 3c668f19e1ce4476b89ff27928db8028, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1701143752647 2023-11-28 03:56:05,097 DEBUG [Listener at localhost/42853] compactions.Compactor(207): Compacting 6dd6971d94d64d53a807de63c42e191f, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1701143754565 2023-11-28 03:56:05,097 DEBUG [Listener at localhost/42853] compactions.Compactor(207): Compacting 12c5607727284891a9470d96da7a3139, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1701143755666 2023-11-28 03:56:05,098 DEBUG [Listener at localhost/42853] compactions.Compactor(207): Compacting 8a3b1e03086d46d7baaa683499b39657, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1701143756809 2023-11-28 03:56:05,098 DEBUG [Listener at localhost/42853] compactions.Compactor(207): Compacting 4de8d1e3f74647ccb14a9dc5a8fcc3e5, keycount=3, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-11-28 03:56:05,105 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/45c07d545bdbc2f92898a848086adde0/family2/52498b831e88442095d4a1cccc76b337, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/45c07d545bdbc2f92898a848086adde0/family2/c8f31136ec6c48f58e1c448f21f608fc, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/45c07d545bdbc2f92898a848086adde0/family2/c3cb4c1d64334b70acd5e8ac92b92396, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/45c07d545bdbc2f92898a848086adde0/family2/85f8ceb20cb54d59a8050b35775fb4cf, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/45c07d545bdbc2f92898a848086adde0/family2/cbdc7b405feb405cb4818757c7a03793] to archive 2023-11-28 03:56:05,106 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-28 03:56:05,106 INFO [Listener at localhost/42853] throttle.PressureAwareThroughputController(145): 6442e6cb874dc41685a9cdfd56a2179e#family1#compaction#201 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-11-28 03:56:05,107 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/45c07d545bdbc2f92898a848086adde0/family2/52498b831e88442095d4a1cccc76b337 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionFromAdmin/45c07d545bdbc2f92898a848086adde0/family2/52498b831e88442095d4a1cccc76b337 2023-11-28 03:56:05,108 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/45c07d545bdbc2f92898a848086adde0/family2/c8f31136ec6c48f58e1c448f21f608fc to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionFromAdmin/45c07d545bdbc2f92898a848086adde0/family2/c8f31136ec6c48f58e1c448f21f608fc 2023-11-28 03:56:05,109 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/45c07d545bdbc2f92898a848086adde0/family2/c3cb4c1d64334b70acd5e8ac92b92396 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionFromAdmin/45c07d545bdbc2f92898a848086adde0/family2/c3cb4c1d64334b70acd5e8ac92b92396 2023-11-28 03:56:05,110 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/45c07d545bdbc2f92898a848086adde0/family2/85f8ceb20cb54d59a8050b35775fb4cf to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionFromAdmin/45c07d545bdbc2f92898a848086adde0/family2/85f8ceb20cb54d59a8050b35775fb4cf 2023-11-28 03:56:05,111 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/45c07d545bdbc2f92898a848086adde0/family2/cbdc7b405feb405cb4818757c7a03793 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionFromAdmin/45c07d545bdbc2f92898a848086adde0/family2/cbdc7b405feb405cb4818757c7a03793 2023-11-28 03:56:05,544 INFO [Listener at localhost/42853] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b20231128fcb4bc19bcc24567a92770cf1228cfbf_del 2023-11-28 03:56:05,544 INFO [Listener at localhost/42853] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20231128fcb4bc19bcc24567a92770cf1228cfbf_del to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20231128fcb4bc19bcc24567a92770cf1228cfbf_del 2023-11-28 03:56:05,605 INFO [Listener at localhost/42853] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 357226ec27bd47babf33efeadde72da0 2023-11-28 03:56:05,606 DEBUG [Listener at localhost/42853] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/6442e6cb874dc41685a9cdfd56a2179e/.tmp/family1/357226ec27bd47babf33efeadde72da0 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/6442e6cb874dc41685a9cdfd56a2179e/family1/357226ec27bd47babf33efeadde72da0 2023-11-28 03:56:05,637 INFO [Listener at localhost/42853] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 357226ec27bd47babf33efeadde72da0 2023-11-28 03:56:05,638 INFO [Listener at localhost/42853] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 6442e6cb874dc41685a9cdfd56a2179e/family1 of 6442e6cb874dc41685a9cdfd56a2179e into 357226ec27bd47babf33efeadde72da0(size=7.0 K), total size for store is 7.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-11-28 03:56:05,638 DEBUG [Listener at localhost/42853] regionserver.HRegion(2289): Compaction status journal for 6442e6cb874dc41685a9cdfd56a2179e: 2023-11-28 03:56:05,638 DEBUG [Listener at localhost/42853] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-11-28 03:56:05,638 DEBUG [Listener at localhost/42853] regionserver.HStore(1912): 6442e6cb874dc41685a9cdfd56a2179e/family2 is initiating major compaction (all files) 2023-11-28 03:56:05,638 INFO [Listener at localhost/42853] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-11-28 03:56:05,638 INFO [Listener at localhost/42853] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-11-28 03:56:05,638 INFO [Listener at localhost/42853] regionserver.HRegion(2259): Starting compaction of 6442e6cb874dc41685a9cdfd56a2179e/family2 in testMajorCompactionFromAdmin,1,1701143750499.6442e6cb874dc41685a9cdfd56a2179e. 2023-11-28 03:56:05,638 INFO [Listener at localhost/42853] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/6442e6cb874dc41685a9cdfd56a2179e/family2/6352c64aff6a4e36bc7588737f37a8bd, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/6442e6cb874dc41685a9cdfd56a2179e/family2/ac838129541241b7bd8bf8d34e279b81, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/6442e6cb874dc41685a9cdfd56a2179e/family2/cb40fdef93ab451aa81e8d1e4efd34be, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/6442e6cb874dc41685a9cdfd56a2179e/family2/1199315641a546cc8b3fc6ee2ec00b6b, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/6442e6cb874dc41685a9cdfd56a2179e/family2/bf321675bb1743878e838aadec3428ba] into tmpdir=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/6442e6cb874dc41685a9cdfd56a2179e/.tmp, totalSize=25.4 K 2023-11-28 03:56:05,639 DEBUG [Listener at localhost/42853] compactions.Compactor(207): Compacting 6352c64aff6a4e36bc7588737f37a8bd, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1701143752647 2023-11-28 03:56:05,639 DEBUG [Listener at localhost/42853] compactions.Compactor(207): Compacting ac838129541241b7bd8bf8d34e279b81, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1701143754565 2023-11-28 03:56:05,640 DEBUG [Listener at localhost/42853] compactions.Compactor(207): Compacting cb40fdef93ab451aa81e8d1e4efd34be, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1701143755666 2023-11-28 03:56:05,640 DEBUG [Listener at localhost/42853] compactions.Compactor(207): Compacting 1199315641a546cc8b3fc6ee2ec00b6b, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1701143756809 2023-11-28 03:56:05,640 DEBUG [Listener at localhost/42853] compactions.Compactor(207): Compacting bf321675bb1743878e838aadec3428ba, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-11-28 03:56:05,656 INFO [Listener at localhost/42853] throttle.PressureAwareThroughputController(145): 6442e6cb874dc41685a9cdfd56a2179e#family2#compaction#202 average throughput is 0.12 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-11-28 03:56:05,707 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/6442e6cb874dc41685a9cdfd56a2179e/family1/3c668f19e1ce4476b89ff27928db8028, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/6442e6cb874dc41685a9cdfd56a2179e/family1/6dd6971d94d64d53a807de63c42e191f, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/6442e6cb874dc41685a9cdfd56a2179e/family1/12c5607727284891a9470d96da7a3139, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/6442e6cb874dc41685a9cdfd56a2179e/family1/8a3b1e03086d46d7baaa683499b39657, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/6442e6cb874dc41685a9cdfd56a2179e/family1/4de8d1e3f74647ccb14a9dc5a8fcc3e5] to archive 2023-11-28 03:56:05,707 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-28 03:56:05,709 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/6442e6cb874dc41685a9cdfd56a2179e/family1/3c668f19e1ce4476b89ff27928db8028 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionFromAdmin/6442e6cb874dc41685a9cdfd56a2179e/family1/3c668f19e1ce4476b89ff27928db8028 2023-11-28 03:56:05,711 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/6442e6cb874dc41685a9cdfd56a2179e/family1/6dd6971d94d64d53a807de63c42e191f to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionFromAdmin/6442e6cb874dc41685a9cdfd56a2179e/family1/6dd6971d94d64d53a807de63c42e191f 2023-11-28 03:56:05,712 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/6442e6cb874dc41685a9cdfd56a2179e/family1/12c5607727284891a9470d96da7a3139 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionFromAdmin/6442e6cb874dc41685a9cdfd56a2179e/family1/12c5607727284891a9470d96da7a3139 2023-11-28 03:56:05,713 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/6442e6cb874dc41685a9cdfd56a2179e/family1/8a3b1e03086d46d7baaa683499b39657 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionFromAdmin/6442e6cb874dc41685a9cdfd56a2179e/family1/8a3b1e03086d46d7baaa683499b39657 2023-11-28 03:56:05,714 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/6442e6cb874dc41685a9cdfd56a2179e/family1/4de8d1e3f74647ccb14a9dc5a8fcc3e5 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionFromAdmin/6442e6cb874dc41685a9cdfd56a2179e/family1/4de8d1e3f74647ccb14a9dc5a8fcc3e5 2023-11-28 03:56:06,070 INFO [Listener at localhost/42853] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b2023112878bda8fcba0a4a17a56028fe6a6dc3dc_del 2023-11-28 03:56:06,070 INFO [Listener at localhost/42853] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023112878bda8fcba0a4a17a56028fe6a6dc3dc_del to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c4ca4238a0b923820dcc509a6f75849b2023112878bda8fcba0a4a17a56028fe6a6dc3dc_del 2023-11-28 03:56:06,490 INFO [Listener at localhost/42853] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for db0a5fa7470c4550a98079df1ffbee2b 2023-11-28 03:56:06,491 DEBUG [Listener at localhost/42853] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/6442e6cb874dc41685a9cdfd56a2179e/.tmp/family2/db0a5fa7470c4550a98079df1ffbee2b as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/6442e6cb874dc41685a9cdfd56a2179e/family2/db0a5fa7470c4550a98079df1ffbee2b 2023-11-28 03:56:06,495 INFO [Listener at localhost/42853] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for db0a5fa7470c4550a98079df1ffbee2b 2023-11-28 03:56:06,496 INFO [Listener at localhost/42853] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 6442e6cb874dc41685a9cdfd56a2179e/family2 of 6442e6cb874dc41685a9cdfd56a2179e into db0a5fa7470c4550a98079df1ffbee2b(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-11-28 03:56:06,497 DEBUG [Listener at localhost/42853] regionserver.HRegion(2289): Compaction status journal for 6442e6cb874dc41685a9cdfd56a2179e: 2023-11-28 03:56:06,501 DEBUG [Listener at localhost/42853] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-11-28 03:56:06,506 DEBUG [Listener at localhost/42853] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-11-28 03:56:06,511 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/6442e6cb874dc41685a9cdfd56a2179e/family2/6352c64aff6a4e36bc7588737f37a8bd, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/6442e6cb874dc41685a9cdfd56a2179e/family2/ac838129541241b7bd8bf8d34e279b81, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/6442e6cb874dc41685a9cdfd56a2179e/family2/cb40fdef93ab451aa81e8d1e4efd34be, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/6442e6cb874dc41685a9cdfd56a2179e/family2/1199315641a546cc8b3fc6ee2ec00b6b, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/6442e6cb874dc41685a9cdfd56a2179e/family2/bf321675bb1743878e838aadec3428ba] to archive 2023-11-28 03:56:06,512 DEBUG [Listener at localhost/42853] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-11-28 03:56:06,512 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-28 03:56:06,514 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/6442e6cb874dc41685a9cdfd56a2179e/family2/6352c64aff6a4e36bc7588737f37a8bd to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionFromAdmin/6442e6cb874dc41685a9cdfd56a2179e/family2/6352c64aff6a4e36bc7588737f37a8bd 2023-11-28 03:56:06,514 DEBUG [Listener at localhost/42853] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-11-28 03:56:06,516 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/6442e6cb874dc41685a9cdfd56a2179e/family2/ac838129541241b7bd8bf8d34e279b81 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionFromAdmin/6442e6cb874dc41685a9cdfd56a2179e/family2/ac838129541241b7bd8bf8d34e279b81 2023-11-28 03:56:06,517 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/6442e6cb874dc41685a9cdfd56a2179e/family2/cb40fdef93ab451aa81e8d1e4efd34be to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionFromAdmin/6442e6cb874dc41685a9cdfd56a2179e/family2/cb40fdef93ab451aa81e8d1e4efd34be 2023-11-28 03:56:06,518 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/6442e6cb874dc41685a9cdfd56a2179e/family2/1199315641a546cc8b3fc6ee2ec00b6b to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionFromAdmin/6442e6cb874dc41685a9cdfd56a2179e/family2/1199315641a546cc8b3fc6ee2ec00b6b 2023-11-28 03:56:06,520 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/6442e6cb874dc41685a9cdfd56a2179e/family2/bf321675bb1743878e838aadec3428ba to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionFromAdmin/6442e6cb874dc41685a9cdfd56a2179e/family2/bf321675bb1743878e838aadec3428ba 2023-11-28 03:56:06,522 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=38593] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10M\x85\x8F\xD8r\xCA\x1B\xD14\xA5[\x19\xC9j\xFDo"\x10G\x04\xCB\x1B\xB0\xAF\xAA\xE5_\xD7\xF9\x11pU\xBF\xC2*\x10\xEB\xAAf\x1C(\xFD\xD1\x9B23\xB2r\x0D\x1Di\xCD2\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}] of the table testMajorCompactionFromAdmin 2023-11-28 03:56:06,547 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] procedure2.ProcedureExecutor(1028): Stored pid=123, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionFromAdmin.mobLock, type=EXCLUSIVE 2023-11-28 03:56:06,548 DEBUG [PEWorker-5] locking.LockProcedure(309): LOCKED pid=123, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionFromAdmin.mobLock, type=EXCLUSIVE 2023-11-28 03:56:06,549 INFO [PEWorker-5] procedure2.TimeoutExecutorThread(81): ADDED pid=123, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionFromAdmin.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1701144366549 2023-11-28 03:56:06,549 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionFromAdmin', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-11-28 03:56:06,550 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-11-28 03:56:06,934 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 3 del files, 12 selected files, and 0 irrelevant files table 'testMajorCompactionFromAdmin' and column 'family1' 2023-11-28 03:56:06,956 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e20231128fc765e646fa04a62a733cbbd7cfd539c_del 2023-11-28 03:56:06,981 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b20231128fcb4bc19bcc24567a92770cf1228cfbf_del 2023-11-28 03:56:07,003 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c20231128cc99133d752c40f5bb896aa9fe2da69e_del 2023-11-28 03:56:07,003 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] compactions.PartitionedMobCompactor(350): After merging, there are 3 del files. table='testMajorCompactionFromAdmin' column='family1' 2023-11-28 03:56:07,004 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] zookeeper.ReadOnlyZKClient(139): Connect 0x4d941a7d to 127.0.0.1:55641 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-11-28 03:56:07,008 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] ipc.AbstractRpcClient(189): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@758a6ac6, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-11-28 03:56:07,009 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20231128 for table='testMajorCompactionFromAdmin' and column='family1' 2023-11-28 03:56:07,961 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20231128ab2a6785e33140259769c83482ded085 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20231128ab2a6785e33140259769c83482ded085 2023-11-28 03:56:07,962 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20231128 table='testMajorCompactionFromAdmin' column='family1' 2023-11-28 03:56:07,972 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(122): Using SIMPLE authentication for service=ClientService, sasl=false 2023-11-28 03:56:07,974 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:53734, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-11-28 03:56:07,982 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(122): Using SIMPLE authentication for service=MasterService, sasl=false 2023-11-28 03:56:07,984 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:46560, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-11-28 03:56:08,012 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/d41d8cd98f00b204e9800998ecf8427e20231128/family1/4c67398ef0164de0885c70e09893b14a first=Optional[0\x00\x00\x00\x01] last=Optional[0\x00\x00\x00\x07] 2023-11-28 03:56:08,013 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionFromAdmin,,1701143750499.b2248d1fbdc8030e0b590971a233e4f2., hostname=jenkins-hbase4.apache.org,41203,1701143600653, seqNum=2 for row with hfile group [{family1,hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/d41d8cd98f00b204e9800998ecf8427e20231128/family1/4c67398ef0164de0885c70e09893b14a}] 2023-11-28 03:56:08,017 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41203] regionserver.HStore(763): Validating hfile at hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/d41d8cd98f00b204e9800998ecf8427e20231128/family1/4c67398ef0164de0885c70e09893b14a for inclusion in b2248d1fbdc8030e0b590971a233e4f2/family1 2023-11-28 03:56:08,038 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41203] regionserver.HStore(775): HFile bounds: first=0\x00\x00\x00\x01 last=0\x00\x00\x00\x07 2023-11-28 03:56:08,038 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41203] regionserver.HStore(777): Region bounds: first= last=1 2023-11-28 03:56:08,038 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41203] regionserver.HRegion(2446): Flush status journal for b2248d1fbdc8030e0b590971a233e4f2: 2023-11-28 03:56:08,038 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41203] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/d41d8cd98f00b204e9800998ecf8427e20231128/family1/4c67398ef0164de0885c70e09893b14a to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testMajorCompactionFromAdmin__5q35lhqhi7snsc6nlaa0gm2b5kt14kppgji8qgquko15riib0dfj3kldsens0c51/family1/4c67398ef0164de0885c70e09893b14a 2023-11-28 03:56:08,041 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41203] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testMajorCompactionFromAdmin__5q35lhqhi7snsc6nlaa0gm2b5kt14kppgji8qgquko15riib0dfj3kldsens0c51/family1/4c67398ef0164de0885c70e09893b14a as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/b2248d1fbdc8030e0b590971a233e4f2/family1/dd0c34b0458340c1bcf23c82db9eab53_SeqId_28_ 2023-11-28 03:56:08,041 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41203] regionserver.HStore(849): Loaded HFile hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testMajorCompactionFromAdmin__5q35lhqhi7snsc6nlaa0gm2b5kt14kppgji8qgquko15riib0dfj3kldsens0c51/family1/4c67398ef0164de0885c70e09893b14a into b2248d1fbdc8030e0b590971a233e4f2/family1 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/b2248d1fbdc8030e0b590971a233e4f2/family1/dd0c34b0458340c1bcf23c82db9eab53_SeqId_28_ - updating store file list. 2023-11-28 03:56:08,064 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41203] regionserver.HStore(882): Loaded HFile hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/b2248d1fbdc8030e0b590971a233e4f2/family1/dd0c34b0458340c1bcf23c82db9eab53_SeqId_28_ into b2248d1fbdc8030e0b590971a233e4f2/family1 2023-11-28 03:56:08,064 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41203] regionserver.HStore(855): Successfully loaded hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testMajorCompactionFromAdmin__5q35lhqhi7snsc6nlaa0gm2b5kt14kppgji8qgquko15riib0dfj3kldsens0c51/family1/4c67398ef0164de0885c70e09893b14a into b2248d1fbdc8030e0b590971a233e4f2/family1 (new location: hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/b2248d1fbdc8030e0b590971a233e4f2/family1/dd0c34b0458340c1bcf23c82db9eab53_SeqId_28_) 2023-11-28 03:56:08,065 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41203] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testMajorCompactionFromAdmin__5q35lhqhi7snsc6nlaa0gm2b5kt14kppgji8qgquko15riib0dfj3kldsens0c51/family1/4c67398ef0164de0885c70e09893b14a 2023-11-28 03:56:08,067 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20231128 table='testMajorCompactionFromAdmin' column='family1' 2023-11-28 03:56:08,068 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-28 03:56:08,069 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202311284bef538b89cc416ba999c2eb9450c187 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202311284bef538b89cc416ba999c2eb9450c187 2023-11-28 03:56:08,070 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e2023112877384403ba994964ba5ad15627e0685f to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e2023112877384403ba994964ba5ad15627e0685f 2023-11-28 03:56:08,071 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20231128ab5a2f26563840708cd763f169fb8949 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20231128ab5a2f26563840708cd763f169fb8949 2023-11-28 03:56:08,072 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20231128f66c6719519a489994bca7c8c780000d to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20231128f66c6719519a489994bca7c8c780000d 2023-11-28 03:56:08,072 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionFromAdmin 4 expired mob files are deleted 2023-11-28 03:56:08,072 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/d41d8cd98f00b204e9800998ecf8427e20231128' 2023-11-28 03:56:08,073 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20231128 for table='testMajorCompactionFromAdmin' and column='family1' 2023-11-28 03:56:08,073 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c81e728d9d4c2f636f067f89cc14862c20231128 for table='testMajorCompactionFromAdmin' and column='family1' 2023-11-28 03:56:08,687 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023112853e48337a5994004b073bcfe235e592c to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c2023112853e48337a5994004b073bcfe235e592c 2023-11-28 03:56:08,688 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20231128 table='testMajorCompactionFromAdmin' column='family1' 2023-11-28 03:56:08,729 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c81e728d9d4c2f636f067f89cc14862c20231128/family1/0d084aaa855b4c26b979a42cbed656f2 first=Optional[2\x00\x00\x00\x01] last=Optional[2\x00\x00\x00\x07] 2023-11-28 03:56:08,730 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionFromAdmin,2,1701143750499.45c07d545bdbc2f92898a848086adde0., hostname=jenkins-hbase4.apache.org,41203,1701143600653, seqNum=2 for row 2 with hfile group [{family1,hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c81e728d9d4c2f636f067f89cc14862c20231128/family1/0d084aaa855b4c26b979a42cbed656f2}] 2023-11-28 03:56:08,731 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41203] regionserver.HStore(763): Validating hfile at hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c81e728d9d4c2f636f067f89cc14862c20231128/family1/0d084aaa855b4c26b979a42cbed656f2 for inclusion in 45c07d545bdbc2f92898a848086adde0/family1 2023-11-28 03:56:08,752 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41203] regionserver.HStore(775): HFile bounds: first=2\x00\x00\x00\x01 last=2\x00\x00\x00\x07 2023-11-28 03:56:08,752 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41203] regionserver.HStore(777): Region bounds: first=2 last= 2023-11-28 03:56:08,752 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41203] regionserver.HRegion(2446): Flush status journal for 45c07d545bdbc2f92898a848086adde0: 2023-11-28 03:56:08,752 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41203] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c81e728d9d4c2f636f067f89cc14862c20231128/family1/0d084aaa855b4c26b979a42cbed656f2 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testMajorCompactionFromAdmin__fo6cbolt3g5573d6bu28j76pecgsqdqffktlh88fi7ggclb2eubobaciudfqu7lb/family1/0d084aaa855b4c26b979a42cbed656f2 2023-11-28 03:56:08,754 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41203] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testMajorCompactionFromAdmin__fo6cbolt3g5573d6bu28j76pecgsqdqffktlh88fi7ggclb2eubobaciudfqu7lb/family1/0d084aaa855b4c26b979a42cbed656f2 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/45c07d545bdbc2f92898a848086adde0/family1/69c1f052a3ea4cf79002004c39efd5de_SeqId_28_ 2023-11-28 03:56:08,754 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41203] regionserver.HStore(849): Loaded HFile hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testMajorCompactionFromAdmin__fo6cbolt3g5573d6bu28j76pecgsqdqffktlh88fi7ggclb2eubobaciudfqu7lb/family1/0d084aaa855b4c26b979a42cbed656f2 into 45c07d545bdbc2f92898a848086adde0/family1 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/45c07d545bdbc2f92898a848086adde0/family1/69c1f052a3ea4cf79002004c39efd5de_SeqId_28_ - updating store file list. 2023-11-28 03:56:08,775 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41203] regionserver.HStore(882): Loaded HFile hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/45c07d545bdbc2f92898a848086adde0/family1/69c1f052a3ea4cf79002004c39efd5de_SeqId_28_ into 45c07d545bdbc2f92898a848086adde0/family1 2023-11-28 03:56:08,775 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41203] regionserver.HStore(855): Successfully loaded hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testMajorCompactionFromAdmin__fo6cbolt3g5573d6bu28j76pecgsqdqffktlh88fi7ggclb2eubobaciudfqu7lb/family1/0d084aaa855b4c26b979a42cbed656f2 into 45c07d545bdbc2f92898a848086adde0/family1 (new location: hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/45c07d545bdbc2f92898a848086adde0/family1/69c1f052a3ea4cf79002004c39efd5de_SeqId_28_) 2023-11-28 03:56:08,776 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41203] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testMajorCompactionFromAdmin__fo6cbolt3g5573d6bu28j76pecgsqdqffktlh88fi7ggclb2eubobaciudfqu7lb/family1/0d084aaa855b4c26b979a42cbed656f2 2023-11-28 03:56:08,778 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20231128 table='testMajorCompactionFromAdmin' column='family1' 2023-11-28 03:56:08,778 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-28 03:56:08,779 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20231128269d1ee93daf4639a2d7afe76c42b6e8 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20231128269d1ee93daf4639a2d7afe76c42b6e8 2023-11-28 03:56:08,780 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202311287219570b249b48c88ca44d108b53d952 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202311287219570b249b48c88ca44d108b53d952 2023-11-28 03:56:08,781 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20231128c1a0e78578c04930ad0ab004ec4b7c02 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20231128c1a0e78578c04930ad0ab004ec4b7c02 2023-11-28 03:56:08,782 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20231128ca05136ff25842999569be376219b42b to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20231128ca05136ff25842999569be376219b42b 2023-11-28 03:56:08,782 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionFromAdmin 4 expired mob files are deleted 2023-11-28 03:56:08,782 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c81e728d9d4c2f636f067f89cc14862c20231128' 2023-11-28 03:56:08,783 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c81e728d9d4c2f636f067f89cc14862c20231128 for table='testMajorCompactionFromAdmin' and column='family1' 2023-11-28 03:56:08,783 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c4ca4238a0b923820dcc509a6f75849b20231128 for table='testMajorCompactionFromAdmin' and column='family1' 2023-11-28 03:56:09,738 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023112818c4cc8dadb5415c9dfa8ec22daa610e to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b2023112818c4cc8dadb5415c9dfa8ec22daa610e 2023-11-28 03:56:09,738 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20231128 table='testMajorCompactionFromAdmin' column='family1' 2023-11-28 03:56:09,780 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c4ca4238a0b923820dcc509a6f75849b20231128/family1/ff35608a63bd4d5ba941f93b491c1ac0 first=Optional[1\x00\x00\x00\x01] last=Optional[1\x00\x00\x00\x07] 2023-11-28 03:56:09,780 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionFromAdmin,1,1701143750499.6442e6cb874dc41685a9cdfd56a2179e., hostname=jenkins-hbase4.apache.org,41203,1701143600653, seqNum=2 for row 1 with hfile group [{family1,hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c4ca4238a0b923820dcc509a6f75849b20231128/family1/ff35608a63bd4d5ba941f93b491c1ac0}] 2023-11-28 03:56:09,782 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41203] regionserver.HStore(763): Validating hfile at hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c4ca4238a0b923820dcc509a6f75849b20231128/family1/ff35608a63bd4d5ba941f93b491c1ac0 for inclusion in 6442e6cb874dc41685a9cdfd56a2179e/family1 2023-11-28 03:56:09,803 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41203] regionserver.HStore(775): HFile bounds: first=1\x00\x00\x00\x01 last=1\x00\x00\x00\x07 2023-11-28 03:56:09,803 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41203] regionserver.HStore(777): Region bounds: first=1 last=2 2023-11-28 03:56:09,803 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41203] regionserver.HRegion(2446): Flush status journal for 6442e6cb874dc41685a9cdfd56a2179e: 2023-11-28 03:56:09,804 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41203] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c4ca4238a0b923820dcc509a6f75849b20231128/family1/ff35608a63bd4d5ba941f93b491c1ac0 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testMajorCompactionFromAdmin__gntp1tme692ps8ha3539ogrb7g9veff1lin2vocci34te591nl5qucairsl016ug/family1/ff35608a63bd4d5ba941f93b491c1ac0 2023-11-28 03:56:09,805 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41203] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testMajorCompactionFromAdmin__gntp1tme692ps8ha3539ogrb7g9veff1lin2vocci34te591nl5qucairsl016ug/family1/ff35608a63bd4d5ba941f93b491c1ac0 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/6442e6cb874dc41685a9cdfd56a2179e/family1/57509a4c51c14b44b0704022116830d8_SeqId_28_ 2023-11-28 03:56:09,806 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41203] regionserver.HStore(849): Loaded HFile hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testMajorCompactionFromAdmin__gntp1tme692ps8ha3539ogrb7g9veff1lin2vocci34te591nl5qucairsl016ug/family1/ff35608a63bd4d5ba941f93b491c1ac0 into 6442e6cb874dc41685a9cdfd56a2179e/family1 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/6442e6cb874dc41685a9cdfd56a2179e/family1/57509a4c51c14b44b0704022116830d8_SeqId_28_ - updating store file list. 2023-11-28 03:56:09,827 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41203] regionserver.HStore(882): Loaded HFile hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/6442e6cb874dc41685a9cdfd56a2179e/family1/57509a4c51c14b44b0704022116830d8_SeqId_28_ into 6442e6cb874dc41685a9cdfd56a2179e/family1 2023-11-28 03:56:09,827 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41203] regionserver.HStore(855): Successfully loaded hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testMajorCompactionFromAdmin__gntp1tme692ps8ha3539ogrb7g9veff1lin2vocci34te591nl5qucairsl016ug/family1/ff35608a63bd4d5ba941f93b491c1ac0 into 6442e6cb874dc41685a9cdfd56a2179e/family1 (new location: hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/6442e6cb874dc41685a9cdfd56a2179e/family1/57509a4c51c14b44b0704022116830d8_SeqId_28_) 2023-11-28 03:56:09,828 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41203] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/staging/jenkins__testMajorCompactionFromAdmin__gntp1tme692ps8ha3539ogrb7g9veff1lin2vocci34te591nl5qucairsl016ug/family1/ff35608a63bd4d5ba941f93b491c1ac0 2023-11-28 03:56:09,829 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20231128 table='testMajorCompactionFromAdmin' column='family1' 2023-11-28 03:56:09,830 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-28 03:56:09,831 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b2023112811bd7bd295fb4b25b693d333431cf856 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b2023112811bd7bd295fb4b25b693d333431cf856 2023-11-28 03:56:09,832 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202311283099f69205b346b6af1c1bb20342fac9 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202311283099f69205b346b6af1c1bb20342fac9 2023-11-28 03:56:09,832 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202311287b9e2b69b5114b1cacb60815355bcd8f to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202311287b9e2b69b5114b1cacb60815355bcd8f 2023-11-28 03:56:09,833 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b2023112892c014b50df043d6ab29ac6d6c8f5436 to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b2023112892c014b50df043d6ab29ac6d6c8f5436 2023-11-28 03:56:09,833 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionFromAdmin 4 expired mob files are deleted 2023-11-28 03:56:09,833 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c4ca4238a0b923820dcc509a6f75849b20231128' 2023-11-28 03:56:09,833 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c4ca4238a0b923820dcc509a6f75849b20231128 for table='testMajorCompactionFromAdmin' and column='family1' 2023-11-28 03:56:09,833 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] compactions.PartitionedMobCompactor(354): After compaction, there are 3 mob files. table='testMajorCompactionFromAdmin' column='family1' 2023-11-28 03:56:09,833 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] compactions.PartitionedMobCompactor(364): After a mob compaction with all files selected, archiving the del files for table='testMajorCompactionFromAdmin' and column='family1' 2023-11-28 03:56:09,834 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] compactions.PartitionedMobCompactor(367): [hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20231128fc765e646fa04a62a733cbbd7cfd539c_del] 2023-11-28 03:56:09,834 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] backup.HFileArchiver(360): Archiving compacted files. 2023-11-28 03:56:09,835 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20231128fc765e646fa04a62a733cbbd7cfd539c_del to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20231128fc765e646fa04a62a733cbbd7cfd539c_del 2023-11-28 03:56:09,835 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] mob.MobUtils(508): Table testMajorCompactionFromAdmin 1 expired mob files are deleted 2023-11-28 03:56:09,835 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] compactions.PartitionedMobCompactor(367): [hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20231128fcb4bc19bcc24567a92770cf1228cfbf_del] 2023-11-28 03:56:09,835 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] backup.HFileArchiver(360): Archiving compacted files. 2023-11-28 03:56:09,837 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20231128fcb4bc19bcc24567a92770cf1228cfbf_del to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20231128fcb4bc19bcc24567a92770cf1228cfbf_del 2023-11-28 03:56:09,837 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] mob.MobUtils(508): Table testMajorCompactionFromAdmin 1 expired mob files are deleted 2023-11-28 03:56:09,837 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] compactions.PartitionedMobCompactor(367): [hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20231128cc99133d752c40f5bb896aa9fe2da69e_del] 2023-11-28 03:56:09,837 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] backup.HFileArchiver(360): Archiving compacted files. 2023-11-28 03:56:09,838 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20231128cc99133d752c40f5bb896aa9fe2da69e_del to hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20231128cc99133d752c40f5bb896aa9fe2da69e_del 2023-11-28 03:56:09,838 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] mob.MobUtils(508): Table testMajorCompactionFromAdmin 1 expired mob files are deleted 2023-11-28 03:56:09,838 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701143603927] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionFromAdmin', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-11-28 03:56:09,839 DEBUG [PEWorker-3] locking.LockProcedure(242): UNLOCKED pid=123, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionFromAdmin.mobLock, type=EXCLUSIVE 2023-11-28 03:56:09,840 INFO [PEWorker-3] procedure2.ProcedureExecutor(1409): Finished pid=123, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionFromAdmin.mobLock, type=EXCLUSIVE in 3.2920 sec 2023-11-28 03:56:09,861 DEBUG [Listener at localhost/42853] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-11-28 03:56:09,863 DEBUG [Listener at localhost/42853] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-11-28 03:56:09,865 DEBUG [Listener at localhost/42853] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-11-28 03:56:09,865 DEBUG [Listener at localhost/42853] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-11-28 03:56:09,869 INFO [Listener at localhost/42853] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-11-28 03:56:09,906 INFO [Listener at localhost/42853] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionFromAdmin Thread=250 (was 263), OpenFileDescriptor=669 (was 687), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=258 (was 204) - SystemLoadAverage LEAK? -, ProcessCount=170 (was 168) - ProcessCount LEAK? -, AvailableMemoryMB=6127 (was 6609) 2023-11-28 03:56:09,906 DEBUG [Listener at localhost/42853] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x041eb7af to 127.0.0.1:55641 2023-11-28 03:56:09,906 DEBUG [Listener at localhost/42853] ipc.AbstractRpcClient(489): Stopping rpc client 2023-11-28 03:56:09,907 INFO [Listener at localhost/42853] hbase.HBaseTestingUtility(1286): Shutting down minicluster 2023-11-28 03:56:09,907 INFO [Listener at localhost/42853] client.ConnectionImplementation(1973): Closing master protocol: MasterService 2023-11-28 03:56:09,907 DEBUG [Listener at localhost/42853] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x79a41f20 to 127.0.0.1:55641 2023-11-28 03:56:09,907 DEBUG [Listener at localhost/42853] ipc.AbstractRpcClient(489): Stopping rpc client 2023-11-28 03:56:09,907 DEBUG [Listener at localhost/42853] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-11-28 03:56:09,907 DEBUG [Listener at localhost/42853] util.JVMClusterUtil(257): Found active master hash=547177689, stopped=false 2023-11-28 03:56:09,907 INFO [Listener at localhost/42853] master.ServerManager(888): Cluster shutdown requested of master=jenkins-hbase4.apache.org,38593,1701143599478 2023-11-28 03:56:09,909 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-11-28 03:56:09,909 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-11-28 03:56:09,909 INFO [Listener at localhost/42853] procedure2.ProcedureExecutor(628): Stopping 2023-11-28 03:56:09,909 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-11-28 03:56:09,909 DEBUG [Listener at localhost/42853] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x3d86cced to 127.0.0.1:55641 2023-11-28 03:56:09,909 DEBUG [Listener at localhost/42853] ipc.AbstractRpcClient(489): Stopping rpc client 2023-11-28 03:56:09,909 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-11-28 03:56:09,910 INFO [Listener at localhost/42853] regionserver.HRegionServer(2300): ***** STOPPING region server 'jenkins-hbase4.apache.org,41203,1701143600653' ***** 2023-11-28 03:56:09,910 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-11-28 03:56:09,910 INFO [Listener at localhost/42853] regionserver.HRegionServer(2314): STOPPED: Shutdown requested 2023-11-28 03:56:09,910 INFO [RS:0;jenkins-hbase4:41203] regionserver.HeapMemoryManager(220): Stopping 2023-11-28 03:56:09,910 INFO [RS:0;jenkins-hbase4:41203] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-11-28 03:56:09,910 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-11-28 03:56:09,910 INFO [RS:0;jenkins-hbase4:41203] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-11-28 03:56:09,911 INFO [RS:0;jenkins-hbase4:41203] regionserver.HRegionServer(3308): Received CLOSE for b2248d1fbdc8030e0b590971a233e4f2 2023-11-28 03:56:09,911 INFO [RS:0;jenkins-hbase4:41203] regionserver.HRegionServer(3308): Received CLOSE for 0758c3f3bb1fb9883b18f53f29d6126b 2023-11-28 03:56:09,911 INFO [RS:0;jenkins-hbase4:41203] regionserver.HRegionServer(3308): Received CLOSE for 227f7071ca3e7b6612f5a149eeb75e98 2023-11-28 03:56:09,911 INFO [RS:0;jenkins-hbase4:41203] regionserver.HRegionServer(3308): Received CLOSE for 8c6dc3114c8460cdf466d087b23e49ef 2023-11-28 03:56:09,911 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing b2248d1fbdc8030e0b590971a233e4f2, disabling compactions & flushes 2023-11-28 03:56:09,911 INFO [RS:0;jenkins-hbase4:41203] regionserver.HRegionServer(3308): Received CLOSE for 8a1f7e59ad225e2d27a98fd6fb53a214 2023-11-28 03:56:09,911 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionFromAdmin,,1701143750499.b2248d1fbdc8030e0b590971a233e4f2. 2023-11-28 03:56:09,911 INFO [RS:0;jenkins-hbase4:41203] regionserver.HRegionServer(3308): Received CLOSE for 303aa5c56b3058a5b63a20fa4b389c99 2023-11-28 03:56:09,911 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionFromAdmin,,1701143750499.b2248d1fbdc8030e0b590971a233e4f2. 2023-11-28 03:56:09,911 INFO [RS:0;jenkins-hbase4:41203] regionserver.HRegionServer(3308): Received CLOSE for 3d7f49de72f2e9e741f04714c6a7648c 2023-11-28 03:56:09,912 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionFromAdmin,,1701143750499.b2248d1fbdc8030e0b590971a233e4f2. after waiting 0 ms 2023-11-28 03:56:09,912 INFO [RS:0;jenkins-hbase4:41203] regionserver.HRegionServer(3308): Received CLOSE for 45c07d545bdbc2f92898a848086adde0 2023-11-28 03:56:09,912 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionFromAdmin,,1701143750499.b2248d1fbdc8030e0b590971a233e4f2. 2023-11-28 03:56:09,912 INFO [RS:0;jenkins-hbase4:41203] regionserver.HRegionServer(3308): Received CLOSE for a2457e9f3652153677ea1cb1286a9784 2023-11-28 03:56:09,912 INFO [RS:0;jenkins-hbase4:41203] regionserver.HRegionServer(3308): Received CLOSE for b3c2fdb68f55a3cbe8d29ee7b6ddd5e5 2023-11-28 03:56:09,912 INFO [RS:0;jenkins-hbase4:41203] regionserver.HRegionServer(3308): Received CLOSE for 8f7c6ece6778b5f8b473ef7daf18a3c4 2023-11-28 03:56:09,912 INFO [RS:0;jenkins-hbase4:41203] regionserver.HRegionServer(3308): Received CLOSE for d38dce7d1890ef61d80a87d7305594c5 2023-11-28 03:56:09,912 INFO [RS:0;jenkins-hbase4:41203] regionserver.HRegionServer(3308): Received CLOSE for 2aad05dc9c73dcbea66259e5c0a6958f 2023-11-28 03:56:09,912 INFO [RS:0;jenkins-hbase4:41203] regionserver.HRegionServer(3308): Received CLOSE for 6442e6cb874dc41685a9cdfd56a2179e 2023-11-28 03:56:09,912 INFO [RS:0;jenkins-hbase4:41203] regionserver.HRegionServer(3308): Received CLOSE for 99b453beb9cd8e4a604b7044c921115b 2023-11-28 03:56:09,912 INFO [RS:0;jenkins-hbase4:41203] regionserver.HRegionServer(3308): Received CLOSE for a075e19a5494ecfa5418f707da74e5f0 2023-11-28 03:56:09,912 INFO [RS:0;jenkins-hbase4:41203] regionserver.HRegionServer(3308): Received CLOSE for 02be414fee3293f77182043b018e4d6b 2023-11-28 03:56:09,912 INFO [RS:0;jenkins-hbase4:41203] regionserver.HRegionServer(3308): Received CLOSE for 2be0907b8bc55af7a11c257a83fc88d6 2023-11-28 03:56:09,912 INFO [RS:0;jenkins-hbase4:41203] regionserver.HRegionServer(3308): Received CLOSE for f40c877e8a9ed20266673cd1013cd07f 2023-11-28 03:56:09,912 INFO [RS:0;jenkins-hbase4:41203] regionserver.HRegionServer(3308): Received CLOSE for 9028932752f3f69b0ced4d93db937aaf 2023-11-28 03:56:09,912 INFO [RS:0;jenkins-hbase4:41203] regionserver.HRegionServer(1147): stopping server jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:56:09,912 DEBUG [RS:0;jenkins-hbase4:41203] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x5a7203e7 to 127.0.0.1:55641 2023-11-28 03:56:09,912 DEBUG [RS:0;jenkins-hbase4:41203] ipc.AbstractRpcClient(489): Stopping rpc client 2023-11-28 03:56:09,913 INFO [RS:0;jenkins-hbase4:41203] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-11-28 03:56:09,913 INFO [RS:0;jenkins-hbase4:41203] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-11-28 03:56:09,913 INFO [RS:0;jenkins-hbase4:41203] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-11-28 03:56:09,913 INFO [RS:0;jenkins-hbase4:41203] regionserver.HRegionServer(3308): Received CLOSE for 1588230740 2023-11-28 03:56:09,913 INFO [RS:0;jenkins-hbase4:41203] regionserver.HRegionServer(1477): Waiting on 21 regions to close 2023-11-28 03:56:09,913 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-11-28 03:56:09,913 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-11-28 03:56:09,913 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-11-28 03:56:09,913 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-11-28 03:56:09,913 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-11-28 03:56:09,913 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 1588230740 3/3 column families, dataSize=50.56 KB heapSize=78.23 KB 2023-11-28 03:56:09,914 DEBUG [RS:0;jenkins-hbase4:41203] regionserver.HRegionServer(1507): Waiting on 02be414fee3293f77182043b018e4d6b, 0758c3f3bb1fb9883b18f53f29d6126b, 1588230740, 227f7071ca3e7b6612f5a149eeb75e98, 2aad05dc9c73dcbea66259e5c0a6958f, 2be0907b8bc55af7a11c257a83fc88d6, 303aa5c56b3058a5b63a20fa4b389c99, 3d7f49de72f2e9e741f04714c6a7648c, 45c07d545bdbc2f92898a848086adde0, 6442e6cb874dc41685a9cdfd56a2179e, 8a1f7e59ad225e2d27a98fd6fb53a214, 8c6dc3114c8460cdf466d087b23e49ef, 8f7c6ece6778b5f8b473ef7daf18a3c4, 9028932752f3f69b0ced4d93db937aaf, 99b453beb9cd8e4a604b7044c921115b, a075e19a5494ecfa5418f707da74e5f0, a2457e9f3652153677ea1cb1286a9784, b2248d1fbdc8030e0b590971a233e4f2, b3c2fdb68f55a3cbe8d29ee7b6ddd5e5, d38dce7d1890ef61d80a87d7305594c5, f40c877e8a9ed20266673cd1013cd07f 2023-11-28 03:56:09,916 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/b2248d1fbdc8030e0b590971a233e4f2/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-11-28 03:56:09,916 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionFromAdmin,,1701143750499.b2248d1fbdc8030e0b590971a233e4f2. 2023-11-28 03:56:09,917 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for b2248d1fbdc8030e0b590971a233e4f2: 2023-11-28 03:56:09,917 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionFromAdmin,,1701143750499.b2248d1fbdc8030e0b590971a233e4f2. 2023-11-28 03:56:09,917 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 0758c3f3bb1fb9883b18f53f29d6126b, disabling compactions & flushes 2023-11-28 03:56:09,917 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMinorCompactionWithWeeklyPolicy,,1701143628658.0758c3f3bb1fb9883b18f53f29d6126b. 2023-11-28 03:56:09,917 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMinorCompactionWithWeeklyPolicy,,1701143628658.0758c3f3bb1fb9883b18f53f29d6126b. 2023-11-28 03:56:09,917 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMinorCompactionWithWeeklyPolicy,,1701143628658.0758c3f3bb1fb9883b18f53f29d6126b. after waiting 0 ms 2023-11-28 03:56:09,917 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMinorCompactionWithWeeklyPolicy,,1701143628658.0758c3f3bb1fb9883b18f53f29d6126b. 2023-11-28 03:56:09,921 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithWeeklyPolicy/0758c3f3bb1fb9883b18f53f29d6126b/recovered.edits/49.seqid, newMaxSeqId=49, maxSeqId=1 2023-11-28 03:56:09,921 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMinorCompactionWithWeeklyPolicy,,1701143628658.0758c3f3bb1fb9883b18f53f29d6126b. 2023-11-28 03:56:09,922 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 0758c3f3bb1fb9883b18f53f29d6126b: 2023-11-28 03:56:09,922 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMinorCompactionWithWeeklyPolicy,,1701143628658.0758c3f3bb1fb9883b18f53f29d6126b. 2023-11-28 03:56:09,922 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 227f7071ca3e7b6612f5a149eeb75e98, disabling compactions & flushes 2023-11-28 03:56:09,922 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region ns:testMinorCompaction,1,1701143722033.227f7071ca3e7b6612f5a149eeb75e98. 2023-11-28 03:56:09,922 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on ns:testMinorCompaction,1,1701143722033.227f7071ca3e7b6612f5a149eeb75e98. 2023-11-28 03:56:09,922 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on ns:testMinorCompaction,1,1701143722033.227f7071ca3e7b6612f5a149eeb75e98. after waiting 0 ms 2023-11-28 03:56:09,922 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region ns:testMinorCompaction,1,1701143722033.227f7071ca3e7b6612f5a149eeb75e98. 2023-11-28 03:56:09,925 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/227f7071ca3e7b6612f5a149eeb75e98/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-11-28 03:56:09,925 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed ns:testMinorCompaction,1,1701143722033.227f7071ca3e7b6612f5a149eeb75e98. 2023-11-28 03:56:09,925 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 227f7071ca3e7b6612f5a149eeb75e98: 2023-11-28 03:56:09,925 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed ns:testMinorCompaction,1,1701143722033.227f7071ca3e7b6612f5a149eeb75e98. 2023-11-28 03:56:09,926 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 8c6dc3114c8460cdf466d087b23e49ef, disabling compactions & flushes 2023-11-28 03:56:09,926 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyPolicy,,1701143645058.8c6dc3114c8460cdf466d087b23e49ef. 2023-11-28 03:56:09,926 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyPolicy,,1701143645058.8c6dc3114c8460cdf466d087b23e49ef. 2023-11-28 03:56:09,926 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyPolicy,,1701143645058.8c6dc3114c8460cdf466d087b23e49ef. after waiting 0 ms 2023-11-28 03:56:09,926 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyPolicy,,1701143645058.8c6dc3114c8460cdf466d087b23e49ef. 2023-11-28 03:56:09,928 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyPolicy/8c6dc3114c8460cdf466d087b23e49ef/recovered.edits/50.seqid, newMaxSeqId=50, maxSeqId=1 2023-11-28 03:56:09,929 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyPolicy,,1701143645058.8c6dc3114c8460cdf466d087b23e49ef. 2023-11-28 03:56:09,929 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 8c6dc3114c8460cdf466d087b23e49ef: 2023-11-28 03:56:09,929 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionWithWeeklyPolicy,,1701143645058.8c6dc3114c8460cdf466d087b23e49ef. 2023-11-28 03:56:09,929 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 8a1f7e59ad225e2d27a98fd6fb53a214, disabling compactions & flushes 2023-11-28 03:56:09,929 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1701143602490.8a1f7e59ad225e2d27a98fd6fb53a214. 2023-11-28 03:56:09,930 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1701143602490.8a1f7e59ad225e2d27a98fd6fb53a214. 2023-11-28 03:56:09,930 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1701143602490.8a1f7e59ad225e2d27a98fd6fb53a214. after waiting 0 ms 2023-11-28 03:56:09,930 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1701143602490.8a1f7e59ad225e2d27a98fd6fb53a214. 2023-11-28 03:56:09,930 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 8a1f7e59ad225e2d27a98fd6fb53a214 1/1 column families, dataSize=109 B heapSize=592 B 2023-11-28 03:56:09,963 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-11-28 03:56:10,009 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: CompactedHFilesCleaner was stopped 2023-11-28 03:56:10,114 DEBUG [RS:0;jenkins-hbase4:41203] regionserver.HRegionServer(1507): Waiting on 02be414fee3293f77182043b018e4d6b, 1588230740, 2aad05dc9c73dcbea66259e5c0a6958f, 2be0907b8bc55af7a11c257a83fc88d6, 303aa5c56b3058a5b63a20fa4b389c99, 3d7f49de72f2e9e741f04714c6a7648c, 45c07d545bdbc2f92898a848086adde0, 6442e6cb874dc41685a9cdfd56a2179e, 8a1f7e59ad225e2d27a98fd6fb53a214, 8f7c6ece6778b5f8b473ef7daf18a3c4, 9028932752f3f69b0ced4d93db937aaf, 99b453beb9cd8e4a604b7044c921115b, a075e19a5494ecfa5418f707da74e5f0, a2457e9f3652153677ea1cb1286a9784, b3c2fdb68f55a3cbe8d29ee7b6ddd5e5, d38dce7d1890ef61d80a87d7305594c5, f40c877e8a9ed20266673cd1013cd07f 2023-11-28 03:56:10,314 DEBUG [RS:0;jenkins-hbase4:41203] regionserver.HRegionServer(1507): Waiting on 02be414fee3293f77182043b018e4d6b, 1588230740, 2aad05dc9c73dcbea66259e5c0a6958f, 2be0907b8bc55af7a11c257a83fc88d6, 303aa5c56b3058a5b63a20fa4b389c99, 3d7f49de72f2e9e741f04714c6a7648c, 45c07d545bdbc2f92898a848086adde0, 6442e6cb874dc41685a9cdfd56a2179e, 8a1f7e59ad225e2d27a98fd6fb53a214, 8f7c6ece6778b5f8b473ef7daf18a3c4, 9028932752f3f69b0ced4d93db937aaf, 99b453beb9cd8e4a604b7044c921115b, a075e19a5494ecfa5418f707da74e5f0, a2457e9f3652153677ea1cb1286a9784, b3c2fdb68f55a3cbe8d29ee7b6ddd5e5, d38dce7d1890ef61d80a87d7305594c5, f40c877e8a9ed20266673cd1013cd07f 2023-11-28 03:56:10,323 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=48.28 KB at sequenceid=130 (bloomFilter=false), to=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/hbase/meta/1588230740/.tmp/info/b98a0ce8ee4d4249b2a47f05f7fb6418 2023-11-28 03:56:10,326 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for b98a0ce8ee4d4249b2a47f05f7fb6418 2023-11-28 03:56:10,338 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=109 B at sequenceid=7 (bloomFilter=true), to=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/hbase/namespace/8a1f7e59ad225e2d27a98fd6fb53a214/.tmp/info/c416b9c4086f4e258ce53e6c952e002e 2023-11-28 03:56:10,341 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/hbase/namespace/8a1f7e59ad225e2d27a98fd6fb53a214/.tmp/info/c416b9c4086f4e258ce53e6c952e002e as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/hbase/namespace/8a1f7e59ad225e2d27a98fd6fb53a214/info/c416b9c4086f4e258ce53e6c952e002e 2023-11-28 03:56:10,344 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/hbase/namespace/8a1f7e59ad225e2d27a98fd6fb53a214/info/c416b9c4086f4e258ce53e6c952e002e, entries=3, sequenceid=7, filesize=4.8 K 2023-11-28 03:56:10,345 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~109 B/109, heapSize ~576 B/576, currentSize=0 B/0 for 8a1f7e59ad225e2d27a98fd6fb53a214 in 415ms, sequenceid=7, compaction requested=false 2023-11-28 03:56:10,348 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/hbase/namespace/8a1f7e59ad225e2d27a98fd6fb53a214/recovered.edits/10.seqid, newMaxSeqId=10, maxSeqId=1 2023-11-28 03:56:10,349 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:namespace,,1701143602490.8a1f7e59ad225e2d27a98fd6fb53a214. 2023-11-28 03:56:10,349 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 8a1f7e59ad225e2d27a98fd6fb53a214: 2023-11-28 03:56:10,349 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1701143602490.8a1f7e59ad225e2d27a98fd6fb53a214. 2023-11-28 03:56:10,350 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 303aa5c56b3058a5b63a20fa4b389c99, disabling compactions & flushes 2023-11-28 03:56:10,350 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testScannerOnBulkLoadRefHFiles,1,1701143640845.303aa5c56b3058a5b63a20fa4b389c99. 2023-11-28 03:56:10,350 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,1,1701143640845.303aa5c56b3058a5b63a20fa4b389c99. 2023-11-28 03:56:10,350 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testScannerOnBulkLoadRefHFiles,1,1701143640845.303aa5c56b3058a5b63a20fa4b389c99. after waiting 0 ms 2023-11-28 03:56:10,350 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testScannerOnBulkLoadRefHFiles,1,1701143640845.303aa5c56b3058a5b63a20fa4b389c99. 2023-11-28 03:56:10,352 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testScannerOnBulkLoadRefHFiles/303aa5c56b3058a5b63a20fa4b389c99/recovered.edits/6.seqid, newMaxSeqId=6, maxSeqId=1 2023-11-28 03:56:10,353 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testScannerOnBulkLoadRefHFiles,1,1701143640845.303aa5c56b3058a5b63a20fa4b389c99. 2023-11-28 03:56:10,353 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 303aa5c56b3058a5b63a20fa4b389c99: 2023-11-28 03:56:10,353 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testScannerOnBulkLoadRefHFiles,1,1701143640845.303aa5c56b3058a5b63a20fa4b389c99. 2023-11-28 03:56:10,353 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 3d7f49de72f2e9e741f04714c6a7648c, disabling compactions & flushes 2023-11-28 03:56:10,353 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,,1701143695226.3d7f49de72f2e9e741f04714c6a7648c. 2023-11-28 03:56:10,353 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,,1701143695226.3d7f49de72f2e9e741f04714c6a7648c. 2023-11-28 03:56:10,353 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,,1701143695226.3d7f49de72f2e9e741f04714c6a7648c. after waiting 0 ms 2023-11-28 03:56:10,353 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,,1701143695226.3d7f49de72f2e9e741f04714c6a7648c. 2023-11-28 03:56:10,356 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/3d7f49de72f2e9e741f04714c6a7648c/recovered.edits/38.seqid, newMaxSeqId=38, maxSeqId=31 2023-11-28 03:56:10,357 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,,1701143695226.3d7f49de72f2e9e741f04714c6a7648c. 2023-11-28 03:56:10,357 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 3d7f49de72f2e9e741f04714c6a7648c: 2023-11-28 03:56:10,357 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testCompactionWithHFileLink,,1701143695226.3d7f49de72f2e9e741f04714c6a7648c. 2023-11-28 03:56:10,357 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 45c07d545bdbc2f92898a848086adde0, disabling compactions & flushes 2023-11-28 03:56:10,357 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionFromAdmin,2,1701143750499.45c07d545bdbc2f92898a848086adde0. 2023-11-28 03:56:10,357 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionFromAdmin,2,1701143750499.45c07d545bdbc2f92898a848086adde0. 2023-11-28 03:56:10,357 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionFromAdmin,2,1701143750499.45c07d545bdbc2f92898a848086adde0. after waiting 0 ms 2023-11-28 03:56:10,357 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionFromAdmin,2,1701143750499.45c07d545bdbc2f92898a848086adde0. 2023-11-28 03:56:10,360 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/45c07d545bdbc2f92898a848086adde0/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-11-28 03:56:10,361 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionFromAdmin,2,1701143750499.45c07d545bdbc2f92898a848086adde0. 2023-11-28 03:56:10,361 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 45c07d545bdbc2f92898a848086adde0: 2023-11-28 03:56:10,361 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionFromAdmin,2,1701143750499.45c07d545bdbc2f92898a848086adde0. 2023-11-28 03:56:10,361 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing a2457e9f3652153677ea1cb1286a9784, disabling compactions & flushes 2023-11-28 03:56:10,361 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testScannerOnBulkLoadRefHFiles,,1701143640845.a2457e9f3652153677ea1cb1286a9784. 2023-11-28 03:56:10,361 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,,1701143640845.a2457e9f3652153677ea1cb1286a9784. 2023-11-28 03:56:10,361 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testScannerOnBulkLoadRefHFiles,,1701143640845.a2457e9f3652153677ea1cb1286a9784. after waiting 0 ms 2023-11-28 03:56:10,361 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testScannerOnBulkLoadRefHFiles,,1701143640845.a2457e9f3652153677ea1cb1286a9784. 2023-11-28 03:56:10,363 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testScannerOnBulkLoadRefHFiles/a2457e9f3652153677ea1cb1286a9784/recovered.edits/6.seqid, newMaxSeqId=6, maxSeqId=1 2023-11-28 03:56:10,364 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testScannerOnBulkLoadRefHFiles,,1701143640845.a2457e9f3652153677ea1cb1286a9784. 2023-11-28 03:56:10,364 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for a2457e9f3652153677ea1cb1286a9784: 2023-11-28 03:56:10,364 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testScannerOnBulkLoadRefHFiles,,1701143640845.a2457e9f3652153677ea1cb1286a9784. 2023-11-28 03:56:10,364 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing b3c2fdb68f55a3cbe8d29ee7b6ddd5e5, disabling compactions & flushes 2023-11-28 03:56:10,364 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,1,1701143695226.b3c2fdb68f55a3cbe8d29ee7b6ddd5e5. 2023-11-28 03:56:10,364 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,1,1701143695226.b3c2fdb68f55a3cbe8d29ee7b6ddd5e5. 2023-11-28 03:56:10,364 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,1,1701143695226.b3c2fdb68f55a3cbe8d29ee7b6ddd5e5. after waiting 0 ms 2023-11-28 03:56:10,364 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,1,1701143695226.b3c2fdb68f55a3cbe8d29ee7b6ddd5e5. 2023-11-28 03:56:10,373 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/b3c2fdb68f55a3cbe8d29ee7b6ddd5e5/recovered.edits/38.seqid, newMaxSeqId=38, maxSeqId=31 2023-11-28 03:56:10,374 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,1,1701143695226.b3c2fdb68f55a3cbe8d29ee7b6ddd5e5. 2023-11-28 03:56:10,374 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for b3c2fdb68f55a3cbe8d29ee7b6ddd5e5: 2023-11-28 03:56:10,374 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testCompactionWithHFileLink,1,1701143695226.b3c2fdb68f55a3cbe8d29ee7b6ddd5e5. 2023-11-28 03:56:10,374 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 8f7c6ece6778b5f8b473ef7daf18a3c4, disabling compactions & flushes 2023-11-28 03:56:10,374 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region ns:testMinorCompaction,,1701143722033.8f7c6ece6778b5f8b473ef7daf18a3c4. 2023-11-28 03:56:10,374 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on ns:testMinorCompaction,,1701143722033.8f7c6ece6778b5f8b473ef7daf18a3c4. 2023-11-28 03:56:10,374 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on ns:testMinorCompaction,,1701143722033.8f7c6ece6778b5f8b473ef7daf18a3c4. after waiting 0 ms 2023-11-28 03:56:10,374 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region ns:testMinorCompaction,,1701143722033.8f7c6ece6778b5f8b473ef7daf18a3c4. 2023-11-28 03:56:10,377 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/8f7c6ece6778b5f8b473ef7daf18a3c4/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-11-28 03:56:10,378 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed ns:testMinorCompaction,,1701143722033.8f7c6ece6778b5f8b473ef7daf18a3c4. 2023-11-28 03:56:10,378 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 8f7c6ece6778b5f8b473ef7daf18a3c4: 2023-11-28 03:56:10,378 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed ns:testMinorCompaction,,1701143722033.8f7c6ece6778b5f8b473ef7daf18a3c4. 2023-11-28 03:56:10,378 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing d38dce7d1890ef61d80a87d7305594c5, disabling compactions & flushes 2023-11-28 03:56:10,378 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region ns:testMinorCompaction,2,1701143722033.d38dce7d1890ef61d80a87d7305594c5. 2023-11-28 03:56:10,378 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on ns:testMinorCompaction,2,1701143722033.d38dce7d1890ef61d80a87d7305594c5. 2023-11-28 03:56:10,378 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on ns:testMinorCompaction,2,1701143722033.d38dce7d1890ef61d80a87d7305594c5. after waiting 0 ms 2023-11-28 03:56:10,378 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region ns:testMinorCompaction,2,1701143722033.d38dce7d1890ef61d80a87d7305594c5. 2023-11-28 03:56:10,382 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/ns/testMinorCompaction/d38dce7d1890ef61d80a87d7305594c5/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-11-28 03:56:10,382 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed ns:testMinorCompaction,2,1701143722033.d38dce7d1890ef61d80a87d7305594c5. 2023-11-28 03:56:10,382 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for d38dce7d1890ef61d80a87d7305594c5: 2023-11-28 03:56:10,382 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed ns:testMinorCompaction,2,1701143722033.d38dce7d1890ef61d80a87d7305594c5. 2023-11-28 03:56:10,382 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 2aad05dc9c73dcbea66259e5c0a6958f, disabling compactions & flushes 2023-11-28 03:56:10,383 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,2,1701143695226.2aad05dc9c73dcbea66259e5c0a6958f. 2023-11-28 03:56:10,383 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,2,1701143695226.2aad05dc9c73dcbea66259e5c0a6958f. 2023-11-28 03:56:10,383 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,2,1701143695226.2aad05dc9c73dcbea66259e5c0a6958f. after waiting 0 ms 2023-11-28 03:56:10,383 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,2,1701143695226.2aad05dc9c73dcbea66259e5c0a6958f. 2023-11-28 03:56:10,388 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testCompactionWithHFileLink/2aad05dc9c73dcbea66259e5c0a6958f/recovered.edits/38.seqid, newMaxSeqId=38, maxSeqId=31 2023-11-28 03:56:10,390 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,2,1701143695226.2aad05dc9c73dcbea66259e5c0a6958f. 2023-11-28 03:56:10,390 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 2aad05dc9c73dcbea66259e5c0a6958f: 2023-11-28 03:56:10,390 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testCompactionWithHFileLink,2,1701143695226.2aad05dc9c73dcbea66259e5c0a6958f. 2023-11-28 03:56:10,390 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 6442e6cb874dc41685a9cdfd56a2179e, disabling compactions & flushes 2023-11-28 03:56:10,390 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionFromAdmin,1,1701143750499.6442e6cb874dc41685a9cdfd56a2179e. 2023-11-28 03:56:10,390 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionFromAdmin,1,1701143750499.6442e6cb874dc41685a9cdfd56a2179e. 2023-11-28 03:56:10,390 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionFromAdmin,1,1701143750499.6442e6cb874dc41685a9cdfd56a2179e. after waiting 0 ms 2023-11-28 03:56:10,390 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionFromAdmin,1,1701143750499.6442e6cb874dc41685a9cdfd56a2179e. 2023-11-28 03:56:10,395 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionFromAdmin/6442e6cb874dc41685a9cdfd56a2179e/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-11-28 03:56:10,396 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionFromAdmin,1,1701143750499.6442e6cb874dc41685a9cdfd56a2179e. 2023-11-28 03:56:10,396 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 6442e6cb874dc41685a9cdfd56a2179e: 2023-11-28 03:56:10,396 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionFromAdmin,1,1701143750499.6442e6cb874dc41685a9cdfd56a2179e. 2023-11-28 03:56:10,396 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 99b453beb9cd8e4a604b7044c921115b, disabling compactions & flushes 2023-11-28 03:56:10,396 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701143673936.99b453beb9cd8e4a604b7044c921115b. 2023-11-28 03:56:10,396 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701143673936.99b453beb9cd8e4a604b7044c921115b. 2023-11-28 03:56:10,396 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701143673936.99b453beb9cd8e4a604b7044c921115b. after waiting 0 ms 2023-11-28 03:56:10,396 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701143673936.99b453beb9cd8e4a604b7044c921115b. 2023-11-28 03:56:10,400 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/99b453beb9cd8e4a604b7044c921115b/recovered.edits/58.seqid, newMaxSeqId=58, maxSeqId=55 2023-11-28 03:56:10,400 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701143673936.99b453beb9cd8e4a604b7044c921115b. 2023-11-28 03:56:10,400 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 99b453beb9cd8e4a604b7044c921115b: 2023-11-28 03:56:10,400 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701143673936.99b453beb9cd8e4a604b7044c921115b. 2023-11-28 03:56:10,401 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing a075e19a5494ecfa5418f707da74e5f0, disabling compactions & flushes 2023-11-28 03:56:10,401 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701143657995.a075e19a5494ecfa5418f707da74e5f0. 2023-11-28 03:56:10,401 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701143657995.a075e19a5494ecfa5418f707da74e5f0. 2023-11-28 03:56:10,401 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701143657995.a075e19a5494ecfa5418f707da74e5f0. after waiting 0 ms 2023-11-28 03:56:10,401 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701143657995.a075e19a5494ecfa5418f707da74e5f0. 2023-11-28 03:56:10,404 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a075e19a5494ecfa5418f707da74e5f0/recovered.edits/58.seqid, newMaxSeqId=58, maxSeqId=55 2023-11-28 03:56:10,404 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701143657995.a075e19a5494ecfa5418f707da74e5f0. 2023-11-28 03:56:10,404 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for a075e19a5494ecfa5418f707da74e5f0: 2023-11-28 03:56:10,404 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701143657995.a075e19a5494ecfa5418f707da74e5f0. 2023-11-28 03:56:10,404 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 02be414fee3293f77182043b018e4d6b, disabling compactions & flushes 2023-11-28 03:56:10,404 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testScannerOnBulkLoadRefHFiles,2,1701143640845.02be414fee3293f77182043b018e4d6b. 2023-11-28 03:56:10,404 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,2,1701143640845.02be414fee3293f77182043b018e4d6b. 2023-11-28 03:56:10,404 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testScannerOnBulkLoadRefHFiles,2,1701143640845.02be414fee3293f77182043b018e4d6b. after waiting 0 ms 2023-11-28 03:56:10,404 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testScannerOnBulkLoadRefHFiles,2,1701143640845.02be414fee3293f77182043b018e4d6b. 2023-11-28 03:56:10,407 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testScannerOnBulkLoadRefHFiles/02be414fee3293f77182043b018e4d6b/recovered.edits/15.seqid, newMaxSeqId=15, maxSeqId=1 2023-11-28 03:56:10,407 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testScannerOnBulkLoadRefHFiles,2,1701143640845.02be414fee3293f77182043b018e4d6b. 2023-11-28 03:56:10,407 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 02be414fee3293f77182043b018e4d6b: 2023-11-28 03:56:10,407 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testScannerOnBulkLoadRefHFiles,2,1701143640845.02be414fee3293f77182043b018e4d6b. 2023-11-28 03:56:10,407 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 2be0907b8bc55af7a11c257a83fc88d6, disabling compactions & flushes 2023-11-28 03:56:10,407 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMinorCompactionWithMonthlyPolicy,,1701143738200.2be0907b8bc55af7a11c257a83fc88d6. 2023-11-28 03:56:10,407 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMinorCompactionWithMonthlyPolicy,,1701143738200.2be0907b8bc55af7a11c257a83fc88d6. 2023-11-28 03:56:10,407 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMinorCompactionWithMonthlyPolicy,,1701143738200.2be0907b8bc55af7a11c257a83fc88d6. after waiting 0 ms 2023-11-28 03:56:10,407 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMinorCompactionWithMonthlyPolicy,,1701143738200.2be0907b8bc55af7a11c257a83fc88d6. 2023-11-28 03:56:10,410 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMinorCompactionWithMonthlyPolicy/2be0907b8bc55af7a11c257a83fc88d6/recovered.edits/52.seqid, newMaxSeqId=52, maxSeqId=1 2023-11-28 03:56:10,411 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMinorCompactionWithMonthlyPolicy,,1701143738200.2be0907b8bc55af7a11c257a83fc88d6. 2023-11-28 03:56:10,411 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 2be0907b8bc55af7a11c257a83fc88d6: 2023-11-28 03:56:10,411 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMinorCompactionWithMonthlyPolicy,,1701143738200.2be0907b8bc55af7a11c257a83fc88d6. 2023-11-28 03:56:10,411 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing f40c877e8a9ed20266673cd1013cd07f, disabling compactions & flushes 2023-11-28 03:56:10,411 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithMonthlyPolicy,,1701143618810.f40c877e8a9ed20266673cd1013cd07f. 2023-11-28 03:56:10,411 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithMonthlyPolicy,,1701143618810.f40c877e8a9ed20266673cd1013cd07f. 2023-11-28 03:56:10,411 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithMonthlyPolicy,,1701143618810.f40c877e8a9ed20266673cd1013cd07f. after waiting 0 ms 2023-11-28 03:56:10,411 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithMonthlyPolicy,,1701143618810.f40c877e8a9ed20266673cd1013cd07f. 2023-11-28 03:56:10,414 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithMonthlyPolicy/f40c877e8a9ed20266673cd1013cd07f/recovered.edits/52.seqid, newMaxSeqId=52, maxSeqId=1 2023-11-28 03:56:10,414 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithMonthlyPolicy,,1701143618810.f40c877e8a9ed20266673cd1013cd07f. 2023-11-28 03:56:10,414 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for f40c877e8a9ed20266673cd1013cd07f: 2023-11-28 03:56:10,414 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionWithMonthlyPolicy,,1701143618810.f40c877e8a9ed20266673cd1013cd07f. 2023-11-28 03:56:10,414 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 9028932752f3f69b0ced4d93db937aaf, disabling compactions & flushes 2023-11-28 03:56:10,414 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthly,,1701143604123.9028932752f3f69b0ced4d93db937aaf. 2023-11-28 03:56:10,414 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1701143604123.9028932752f3f69b0ced4d93db937aaf. 2023-11-28 03:56:10,414 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1701143604123.9028932752f3f69b0ced4d93db937aaf. after waiting 0 ms 2023-11-28 03:56:10,415 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthly,,1701143604123.9028932752f3f69b0ced4d93db937aaf. 2023-11-28 03:56:10,417 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9028932752f3f69b0ced4d93db937aaf/recovered.edits/55.seqid, newMaxSeqId=55, maxSeqId=50 2023-11-28 03:56:10,417 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthly,,1701143604123.9028932752f3f69b0ced4d93db937aaf. 2023-11-28 03:56:10,417 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 9028932752f3f69b0ced4d93db937aaf: 2023-11-28 03:56:10,417 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionWithWeeklyFollowedByMonthly,,1701143604123.9028932752f3f69b0ced4d93db937aaf. 2023-11-28 03:56:10,514 DEBUG [RS:0;jenkins-hbase4:41203] regionserver.HRegionServer(1507): Waiting on 1588230740 2023-11-28 03:56:10,715 DEBUG [RS:0;jenkins-hbase4:41203] regionserver.HRegionServer(1507): Waiting on 1588230740 2023-11-28 03:56:10,732 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=102 B at sequenceid=130 (bloomFilter=false), to=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/hbase/meta/1588230740/.tmp/rep_barrier/2052349e6dcd467d81ee7769bfe5b3a4 2023-11-28 03:56:10,737 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2052349e6dcd467d81ee7769bfe5b3a4 2023-11-28 03:56:10,888 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: MemstoreFlusherChore was stopped 2023-11-28 03:56:10,888 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: CompactionChecker was stopped 2023-11-28 03:56:10,915 INFO [RS:0;jenkins-hbase4:41203] regionserver.HRegionServer(1477): Waiting on 1 regions to close 2023-11-28 03:56:10,915 DEBUG [RS:0;jenkins-hbase4:41203] regionserver.HRegionServer(1481): Online Regions={1588230740=hbase:meta,,1.1588230740} 2023-11-28 03:56:10,915 DEBUG [RS:0;jenkins-hbase4:41203] regionserver.HRegionServer(1507): Waiting on 1588230740 2023-11-28 03:56:11,115 DEBUG [RS:0;jenkins-hbase4:41203] regionserver.HRegionServer(1507): Waiting on 1588230740 2023-11-28 03:56:11,143 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.18 KB at sequenceid=130 (bloomFilter=false), to=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/hbase/meta/1588230740/.tmp/table/5e52bfcdbe9441e3a5be08a0b7dcf03d 2023-11-28 03:56:11,147 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 5e52bfcdbe9441e3a5be08a0b7dcf03d 2023-11-28 03:56:11,148 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/hbase/meta/1588230740/.tmp/info/b98a0ce8ee4d4249b2a47f05f7fb6418 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/hbase/meta/1588230740/info/b98a0ce8ee4d4249b2a47f05f7fb6418 2023-11-28 03:56:11,152 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for b98a0ce8ee4d4249b2a47f05f7fb6418 2023-11-28 03:56:11,152 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/hbase/meta/1588230740/info/b98a0ce8ee4d4249b2a47f05f7fb6418, entries=226, sequenceid=130, filesize=37.9 K 2023-11-28 03:56:11,153 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/hbase/meta/1588230740/.tmp/rep_barrier/2052349e6dcd467d81ee7769bfe5b3a4 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/hbase/meta/1588230740/rep_barrier/2052349e6dcd467d81ee7769bfe5b3a4 2023-11-28 03:56:11,155 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2052349e6dcd467d81ee7769bfe5b3a4 2023-11-28 03:56:11,155 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/hbase/meta/1588230740/rep_barrier/2052349e6dcd467d81ee7769bfe5b3a4, entries=1, sequenceid=130, filesize=5.0 K 2023-11-28 03:56:11,156 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/hbase/meta/1588230740/.tmp/table/5e52bfcdbe9441e3a5be08a0b7dcf03d as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/hbase/meta/1588230740/table/5e52bfcdbe9441e3a5be08a0b7dcf03d 2023-11-28 03:56:11,158 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 5e52bfcdbe9441e3a5be08a0b7dcf03d 2023-11-28 03:56:11,159 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/hbase/meta/1588230740/table/5e52bfcdbe9441e3a5be08a0b7dcf03d, entries=27, sequenceid=130, filesize=6.6 K 2023-11-28 03:56:11,160 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~50.56 KB/51772, heapSize ~78.19 KB/80064, currentSize=0 B/0 for 1588230740 in 1247ms, sequenceid=130, compaction requested=false 2023-11-28 03:56:11,164 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/data/hbase/meta/1588230740/recovered.edits/133.seqid, newMaxSeqId=133, maxSeqId=1 2023-11-28 03:56:11,164 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-11-28 03:56:11,164 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-11-28 03:56:11,164 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-11-28 03:56:11,164 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-11-28 03:56:11,315 INFO [RS:0;jenkins-hbase4:41203] regionserver.HRegionServer(1173): stopping server jenkins-hbase4.apache.org,41203,1701143600653; all regions closed. 2023-11-28 03:56:11,323 DEBUG [RS:0;jenkins-hbase4:41203] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/oldWALs 2023-11-28 03:56:11,323 INFO [RS:0;jenkins-hbase4:41203] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C41203%2C1701143600653.meta:.meta(num 1701143602226) 2023-11-28 03:56:11,327 DEBUG [RS:0;jenkins-hbase4:41203] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/oldWALs 2023-11-28 03:56:11,327 INFO [RS:0;jenkins-hbase4:41203] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C41203%2C1701143600653:(num 1701143602025) 2023-11-28 03:56:11,327 DEBUG [RS:0;jenkins-hbase4:41203] ipc.AbstractRpcClient(489): Stopping rpc client 2023-11-28 03:56:11,327 INFO [RS:0;jenkins-hbase4:41203] regionserver.LeaseManager(133): Closed leases 2023-11-28 03:56:11,327 INFO [RS:0;jenkins-hbase4:41203] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-11-28 03:56:11,328 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-11-28 03:56:11,329 INFO [RS:0;jenkins-hbase4:41203] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:41203 2023-11-28 03:56:11,338 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,41203,1701143600653 2023-11-28 03:56:11,338 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-11-28 03:56:11,338 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-11-28 03:56:11,340 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,41203,1701143600653] 2023-11-28 03:56:11,340 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,41203,1701143600653; numProcessing=1 2023-11-28 03:56:11,341 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,41203,1701143600653 already deleted, retry=false 2023-11-28 03:56:11,341 INFO [RegionServerTracker-0] master.ServerManager(554): Cluster shutdown set; jenkins-hbase4.apache.org,41203,1701143600653 expired; onlineServers=0 2023-11-28 03:56:11,341 INFO [RegionServerTracker-0] regionserver.HRegionServer(2300): ***** STOPPING region server 'jenkins-hbase4.apache.org,38593,1701143599478' ***** 2023-11-28 03:56:11,341 INFO [RegionServerTracker-0] regionserver.HRegionServer(2314): STOPPED: Cluster shutdown set; onlineServer=0 2023-11-28 03:56:11,342 DEBUG [M:0;jenkins-hbase4:38593] ipc.AbstractRpcClient(189): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@61a04ee2, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-11-28 03:56:11,342 INFO [M:0;jenkins-hbase4:38593] regionserver.HRegionServer(1147): stopping server jenkins-hbase4.apache.org,38593,1701143599478 2023-11-28 03:56:11,342 INFO [M:0;jenkins-hbase4:38593] regionserver.HRegionServer(1173): stopping server jenkins-hbase4.apache.org,38593,1701143599478; all regions closed. 2023-11-28 03:56:11,342 DEBUG [M:0;jenkins-hbase4:38593] ipc.AbstractRpcClient(489): Stopping rpc client 2023-11-28 03:56:11,342 DEBUG [M:0;jenkins-hbase4:38593] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-11-28 03:56:11,342 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-11-28 03:56:11,342 DEBUG [M:0;jenkins-hbase4:38593] cleaner.HFileCleaner(317): Stopping file delete threads 2023-11-28 03:56:11,342 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1701143601728] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1701143601728,5,FailOnTimeoutGroup] 2023-11-28 03:56:11,342 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1701143601729] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1701143601729,5,FailOnTimeoutGroup] 2023-11-28 03:56:11,342 INFO [M:0;jenkins-hbase4:38593] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-11-28 03:56:11,342 INFO [M:0;jenkins-hbase4:38593] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-11-28 03:56:11,342 INFO [M:0;jenkins-hbase4:38593] hbase.ChoreService(369): Chore service for: master/jenkins-hbase4:0 had [] on shutdown 2023-11-28 03:56:11,343 DEBUG [M:0;jenkins-hbase4:38593] master.HMaster(1512): Stopping service threads 2023-11-28 03:56:11,343 INFO [M:0;jenkins-hbase4:38593] procedure2.RemoteProcedureDispatcher(119): Stopping procedure remote dispatcher 2023-11-28 03:56:11,343 ERROR [M:0;jenkins-hbase4:38593] procedure2.ProcedureExecutor(652): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-20,5,PEWorkerGroup] Thread[HFileArchiver-21,5,PEWorkerGroup] Thread[HFileArchiver-22,5,PEWorkerGroup] 2023-11-28 03:56:11,343 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/master 2023-11-28 03:56:11,343 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-11-28 03:56:11,344 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-11-28 03:56:11,344 INFO [M:0;jenkins-hbase4:38593] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-11-28 03:56:11,344 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-11-28 03:56:11,344 DEBUG [M:0;jenkins-hbase4:38593] zookeeper.ZKUtil(399): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Unable to get data of znode /hbase/master because node does not exist (not an error) 2023-11-28 03:56:11,344 WARN [M:0;jenkins-hbase4:38593] master.ActiveMasterManager(326): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-11-28 03:56:11,344 INFO [M:0;jenkins-hbase4:38593] assignment.AssignmentManager(315): Stopping assignment manager 2023-11-28 03:56:11,345 INFO [M:0;jenkins-hbase4:38593] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-11-28 03:56:11,345 DEBUG [M:0;jenkins-hbase4:38593] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-11-28 03:56:11,345 INFO [M:0;jenkins-hbase4:38593] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-11-28 03:56:11,345 DEBUG [M:0;jenkins-hbase4:38593] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-11-28 03:56:11,345 DEBUG [M:0;jenkins-hbase4:38593] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-11-28 03:56:11,345 DEBUG [M:0;jenkins-hbase4:38593] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-11-28 03:56:11,345 INFO [M:0;jenkins-hbase4:38593] regionserver.HRegion(2745): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=512.79 KB heapSize=598.59 KB 2023-11-28 03:56:11,440 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-11-28 03:56:11,440 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): regionserver:41203-0x100321f3a150001, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-11-28 03:56:11,440 INFO [RS:0;jenkins-hbase4:41203] regionserver.HRegionServer(1230): Exiting; stopping=jenkins-hbase4.apache.org,41203,1701143600653; zookeeper connection closed. 2023-11-28 03:56:11,440 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@72244cc4] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@72244cc4 2023-11-28 03:56:11,441 INFO [Listener at localhost/42853] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 1 regionserver(s) complete 2023-11-28 03:56:11,753 INFO [M:0;jenkins-hbase4:38593] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=512.79 KB at sequenceid=1006 (bloomFilter=true), to=hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/2c3df67d430c42fb9ba87f608d1e8758 2023-11-28 03:56:11,756 INFO [M:0;jenkins-hbase4:38593] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2c3df67d430c42fb9ba87f608d1e8758 2023-11-28 03:56:11,756 DEBUG [M:0;jenkins-hbase4:38593] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/2c3df67d430c42fb9ba87f608d1e8758 as hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/2c3df67d430c42fb9ba87f608d1e8758 2023-11-28 03:56:11,759 INFO [M:0;jenkins-hbase4:38593] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2c3df67d430c42fb9ba87f608d1e8758 2023-11-28 03:56:11,759 INFO [M:0;jenkins-hbase4:38593] regionserver.HStore(1080): Added hdfs://localhost:34103/user/jenkins/test-data/f9d9b268-d568-5b8f-9691-e62e2c0c3adb/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/2c3df67d430c42fb9ba87f608d1e8758, entries=123, sequenceid=1006, filesize=33.3 K 2023-11-28 03:56:11,760 INFO [M:0;jenkins-hbase4:38593] regionserver.HRegion(2948): Finished flush of dataSize ~512.79 KB/525092, heapSize ~598.58 KB/612944, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 415ms, sequenceid=1006, compaction requested=false 2023-11-28 03:56:11,760 INFO [M:0;jenkins-hbase4:38593] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-11-28 03:56:11,760 DEBUG [M:0;jenkins-hbase4:38593] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-11-28 03:56:11,763 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-11-28 03:56:11,763 INFO [M:0;jenkins-hbase4:38593] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-11-28 03:56:11,764 INFO [M:0;jenkins-hbase4:38593] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:38593 2023-11-28 03:56:11,767 DEBUG [M:0;jenkins-hbase4:38593] zookeeper.RecoverableZooKeeper(172): Node /hbase/rs/jenkins-hbase4.apache.org,38593,1701143599478 already deleted, retry=false 2023-11-28 03:56:11,869 INFO [M:0;jenkins-hbase4:38593] regionserver.HRegionServer(1230): Exiting; stopping=jenkins-hbase4.apache.org,38593,1701143599478; zookeeper connection closed. 2023-11-28 03:56:11,869 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-11-28 03:56:11,869 DEBUG [Listener at localhost/42853-EventThread] zookeeper.ZKWatcher(600): master:38593-0x100321f3a150000, quorum=127.0.0.1:55641, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-11-28 03:56:11,871 WARN [Listener at localhost/42853] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-11-28 03:56:11,873 INFO [Listener at localhost/42853] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-11-28 03:56:11,980 WARN [BP-1757459218-172.31.14.131-1701143595945 heartbeating to localhost/127.0.0.1:34103] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-11-28 03:56:11,980 WARN [BP-1757459218-172.31.14.131-1701143595945 heartbeating to localhost/127.0.0.1:34103] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1757459218-172.31.14.131-1701143595945 (Datanode Uuid 341a4277-f306-4f80-b29e-d311107d0264) service to localhost/127.0.0.1:34103 2023-11-28 03:56:11,982 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/2f2fe2ad-19b7-95b4-b267-35017f394f29/cluster_c657509c-448c-2cac-5018-4e5cd83a6dc3/dfs/data/data1/current/BP-1757459218-172.31.14.131-1701143595945] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-11-28 03:56:11,982 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/2f2fe2ad-19b7-95b4-b267-35017f394f29/cluster_c657509c-448c-2cac-5018-4e5cd83a6dc3/dfs/data/data2/current/BP-1757459218-172.31.14.131-1701143595945] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-11-28 03:56:12,008 INFO [Listener at localhost/42853] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-11-28 03:56:12,141 INFO [Listener at localhost/42853] zookeeper.MiniZooKeeperCluster(344): Shutdown MiniZK cluster with all ZK servers 2023-11-28 03:56:12,224 INFO [Listener at localhost/42853] hbase.HBaseTestingUtility(1293): Minicluster is down